CN116756225B - Situation data information processing method based on computer network security - Google Patents
Situation data information processing method based on computer network security Download PDFInfo
- Publication number
- CN116756225B CN116756225B CN202311019142.8A CN202311019142A CN116756225B CN 116756225 B CN116756225 B CN 116756225B CN 202311019142 A CN202311019142 A CN 202311019142A CN 116756225 B CN116756225 B CN 116756225B
- Authority
- CN
- China
- Prior art keywords
- data
- unit
- network security
- situation
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 19
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 68
- 238000001514 detection method Methods 0.000 claims abstract description 35
- 230000002159 abnormal effect Effects 0.000 claims abstract description 29
- 238000011156 evaluation Methods 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 18
- 230000005856 abnormality Effects 0.000 claims abstract description 6
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000012800 visualization Methods 0.000 claims abstract description 4
- 238000006243 chemical reaction Methods 0.000 claims description 32
- 238000000034 method Methods 0.000 claims description 31
- 210000002569 neuron Anatomy 0.000 claims description 27
- 238000011084 recovery Methods 0.000 claims description 27
- 230000006870 function Effects 0.000 claims description 26
- 238000003860 storage Methods 0.000 claims description 18
- 230000004913 activation Effects 0.000 claims description 15
- 230000009466 transformation Effects 0.000 claims description 14
- 238000004458 analytical method Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000004140 cleaning Methods 0.000 claims description 12
- 238000012216 screening Methods 0.000 claims description 12
- 230000005540 biological transmission Effects 0.000 claims description 11
- 238000002156 mixing Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 231100000279 safety data Toxicity 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 239000000463 material Substances 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 4
- 239000012528 membrane Substances 0.000 description 4
- 238000013480 data collection Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000001208 nuclear magnetic resonance pulse sequence Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000946 synaptic effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Quality & Reliability (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a situation data information processing method based on computer network security, which relates to the technical field of data information processing and mainly solves the problem of computer network situation data information processing; dividing the acquired network security data; preprocessing the network security data after division; classifying and storing the ordered network security data; adopting a corresponding processing mode to detect and recover data errors; carrying out feedback evaluation and visualization on the network security situation data processing result and the computing node monitoring result; the network security data is classified by the clustering hybrid algorithm, and the abnormal threat model detects the abnormality of the network security data and the computing nodes, so that the identification capability of network information is greatly improved, the detection efficiency of the network security situation data is accelerated, and the cost of manpower and material resources is greatly reduced.
Description
Technical Field
The invention relates to the technical field of data information processing, in particular to a situation data information processing method based on computer network security.
Background
At present, more new technologies related to big data are mainly reflected in the aspects of human social activities, data information safety, production industry and the like in rapid development, can better serve the society, and have profound effects. The scale of data is gradually expanded, the realization of data sharing becomes pursuit of enterprises, and in the increasingly growing demands of people, some emerging industries can appear in the fields of view of people, so that the employment problem of partial people is solved, and the social resources are further expanded. With the widespread spread of information, security privacy has also become a focus of attention, which is an unavoidable existence under information sharing.
In a specific application, a computer network data packet typically includes the following data elements:
IP address: each packet has a unique IP address identifying the source address of the packet. Port number: both the source address and destination address of the packet require a port number for identifying the host at which the source address and destination address are located.
Protocol type: for identifying the type of protocol used by the data packet, such as TCP or UDP.
Length: the length used to identify the data packet is typically in units of 4 bytes.
Data: the content of the data packet includes information such as source address, destination address, protocol type, length, etc.
An identifier: an identifier, such as an identifier, header field, etc., for identifying the data packet.
And (3) checksum: the method is used for checking the integrity of the data packet and preventing the data packet from being tampered or lost.
Source address: the source address for identifying the data packet is typically expressed in the form of an IP address.
Target address: the destination address for identifying the data packet is typically expressed in the form of an IP address.
These data elements constitute a data packet for transmission over the network. The above elements are all important components that make up situation data. How to realize situation data information processing in the network data information transmission process, how to play a critical role in the data parameters, how to screen and integrate a large amount of data are problems to be solved urgently,
the generation of computer network data information is usually accompanied by a large amount of redundant data, so that the analysis of the network information data becomes very difficult, and the whole network security loophole can be leaked due to the consequences caused by abnormal data or abnormal computing nodes, so that the intelligent degree is low in the process of computer network security sensing, computing, processing and analyzing in the prior art, but the processing of data batch information is difficult to realize when massive data information appears. The processing of network data and the searching of abnormal computing nodes in the prior art often cause great effort.
Disclosure of Invention
Aiming at the defects of the technology, the invention discloses a situation data information processing method based on computer network security, which converts security event information into a machine receiving form through a neural network converter, classifies network security data through a clustering hybrid algorithm, detects network security data and computing node abnormality through an abnormal threat model, greatly improves network information identification capability, accelerates network security situation data detection efficiency, and greatly reduces manpower and material resource cost.
In view of this, the present invention provides a situation data information processing method based on computer network security, comprising the following steps,
step 1, acquiring network security situation data of various types;
acquiring multiple types of network security data through a data acquisition module, wherein the data acquisition module comprises a content acquisition unit and a link filtering unit, the content acquisition unit is used for acquiring multiple types of network security data content, and the link filtering unit is used for removing network links of the acquired network security data;
step 2, dividing the acquired network security data;
dividing the acquired network security data into a plurality of data blocks by adopting a data dividing module;
step 3, preprocessing the network security data after division;
the method comprises the steps that a data processing module is adopted to preprocess divided network security data, the data processing module comprises a main controller, a data conversion unit, a data cleaning unit and a data sorting unit, the main controller is used for adjusting the working states of all modules of the situation data processing system, the data conversion unit is used for converting network security situation information into a machine receiving form through a neural network converter, the data cleaning unit comprises a filling subunit and a denoising subunit, the filling subunit fills up a data incomplete part through an interpolation algorithm, the denoising subunit is used for removing redundant parts of data through a wavelet transformation algorithm, the data sorting unit is used for sorting the cleaned data into a sequence according to time, the output end of the data conversion unit is connected with the input end of the data cleaning unit, and the output end of the data cleaning unit is connected with the input end of the data sorting unit;
step 4, classifying and storing the ordered network security data;
classifying the network security data by adopting a data classification module, wherein the data classification module comprises a discrimination unit and a storage unit, the discrimination unit is used for discriminating the network security data according to time and situation by a clustering and mixing algorithm, the storage unit is used for carrying out distributed storage on each type of network security data, and the output end of the discrimination unit is connected with the input end of the storage unit;
the working method of the clustering mixing algorithm comprises the following steps: firstly, performing dimension reduction and mapping on high-dimension data by using a self-organizing map algorithm to obtain a self-organizing map, then taking nodes on the self-organizing map as initial clusters, then executing a K-means clustering algorithm on each cluster to divide the clusters into sub-clusters, then calculating the distance between each sub-cluster and other sub-clusters and the initial clusters, finally merging the sub-clusters by adopting split hierarchical clusters until the number of the clusters reaches saturation, then executing a K-means algorithm on all the clusters to obtain mass centers, and representing the clusters by the mass centers;
step 5, carrying out data error detection and recovery on the classified network security data by adopting a corresponding processing mode;
the method comprises the steps that error detection and recovery of network safety data are carried out through a detection recovery module, the detection recovery module comprises a detection unit, an analysis unit, a screening unit, a scheduling unit and a recovery unit, the detection unit is used for carrying out abnormal detection on classified network safety data through an abnormal threat model, the analysis unit is used for discovering abnormal reasons of detection, the scheduling unit is used for automatically distributing and scheduling computing nodes through a scheduling algorithm to detect abnormal threats, the screening unit is used for monitoring each computing node through a search engine and isolating the abnormal computing nodes, the recovery unit is used for recovering the computing nodes monitored for abnormality through restarting and self-checking, the output end of the detection unit is connected with the input end of the analysis unit, the output end of the analysis unit is connected with the input end of the scheduling unit, the output end of the scheduling unit is connected with the input end of the screening unit, and the output end of the screening unit is connected with the input end of the recovery unit;
step 6, carrying out feedback evaluation and visualization on the network security situation data processing result and the computing node monitoring result;
the intelligent display module is arranged to visually display network security situation data processing results, computing node monitoring results and evaluation feedback results thereof and generate corresponding graphic reports, the intelligent display module comprises a display unit, an evaluation unit and a wireless transmission unit, the display unit displays the data processing results and the computing node monitoring results through a touch screen, the evaluation unit evaluates and feeds back the data processing results and the computing node monitoring results through an evaluation matching algorithm, the wireless transmission unit performs wireless communication with a plurality of terminals through a TCP/IP protocol, the output end of the evaluation unit is connected with the input end of the display unit, and the output end of the display unit is connected with the input end of the wireless transmission unit.
As a further embodiment of the present invention, the neural network converter includes a preceding neuron, a preset information conversion algorithm, a preset pulse conversion algorithm, and a following neuron, where the preceding neuron receives network security situation information and linearly converts the network security situation information into encoded pulse input information by using a preset information conversion method, and then non-linearly converts the encoded pulse input information into neuron information by using a preset pulse conversion method and outputs data from the following neuron.
As a further embodiment of the invention, the main controller comprises an FPGA+DSP processing module, the DSP processing module is an ATMega328 type acquisition chip, the DSP processing module integrates a 14-path GPIO interface, a 6-path PWM interface, a 12-bit ADC interface, a UART serial port, a 1-path SPI interface and a 1-path I2C interface, and the FPGA processing module is a Spartan-7 series XC7S15-2CSGA225I chip.
As a further embodiment of the present invention, the abnormal threat model includes a loading unit, a calculating unit, a learning unit and an identifying unit, where the loading unit reads data from the storage unit through a language describing a data stream and translates the data stream into data to be executed, the calculating unit adjusts large batches of data to be executed in real time according to a weight value and a threshold value, and then performs nonlinear transformation on a calculation result through an activation function to extract complex features, the learning unit continuously adjusts the weight and the threshold value in the neural network according to a preset objective function or an error function to iterate the execution data of the training process and adjust the basic network model, the identifying unit predicts abnormal threat types of the classified data by adopting different activation functions and objective functions, an output end of the loading unit is connected to an input end of the calculating unit, an output end of the calculating unit is connected to an input end of the learning unit, and an output end of the learning unit is connected to an input end of the identifying unit.
As a further embodiment of the present invention, the working method of the computing unit is as follows: firstly, verifying the format of a real-time data stream, dividing stream data into a plurality of data blocks according to the length, the position and the batch size of the data, distributing the data blocks to different computing nodes for execution, loading and converting the data into elastic distributed key value pairs suitable for mapping task reading, matching according to similar characteristics between a source domain and a target domain, outputting a series of key words and key value pairs as intermediate results, partitioning, sorting, merging and merging the key words and the key values, then delivering the key words and the key value pairs to corresponding simplification tasks for parallel processing on a plurality of machines, and summarizing and calculating a key value list with the same key words by the same simplification task, thereby executing logic output and storing the results.
As a further embodiment of the present invention, the learning unit performs feature transformation by using 3×3 and 5×5 convolutions, performs feature downsampling by using a maximum pooling layer and performs feature processing by using 1×1 convolutions, performs classification processing by using three nonlinear layers, and finally improves training performance by using a combined activation function, where the combined activation function is:
in the formula (1), n is the stacking number, a i B for scaling parameters i A (x) is a combined activation function, x is input data, lambdaIs a super parameter.
As a further embodiment of the present invention, the working method of the loading unit is as follows: firstly, receiving a request, acquiring data from a storage unit, then analyzing and processing the acquired data, translating a data stream into executable data, performing code conversion and character set processing on text data, performing byte order conversion and data type conversion on binary data, and finally taking the translated data as the data to be executed.
Has the positive beneficial effects that:
the invention discloses a situation data information processing method based on computer network security, which classifies network security data through a clustering hybrid algorithm, and an abnormal threat model detects the abnormality of the network security data and a computing node, so that the identification capability of network information is greatly improved, the detection efficiency of the network security situation data is accelerated, and the cost of manpower and material resources is greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings which are required in the description of the embodiments or the prior art will be briefly described below, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings may be obtained from these drawings without inventive faculty for a person skilled in the art, wherein,
figure 1 is a flow chart of the present invention,
figure 2 is a block diagram of the modules employed in the present invention,
figure 3 is a diagram of a test recovery module architecture,
figure 4 is a diagram of a data processing module architecture,
fig. 5 is an abnormal threat model architecture diagram.
Detailed Description
The following description of the embodiments of the present disclosure will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the disclosure. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.
As shown in fig. 1-5, a situation data information processing method based on computer network security includes the steps of,
step 1, acquiring network security situation data of various types;
acquiring multiple types of network security data through a data acquisition module, wherein the data acquisition module comprises a content acquisition unit and a link filtering unit, the content acquisition unit is used for acquiring multiple types of network security data content, and the link filtering unit is used for removing network links of the acquired network security data;
step 2, dividing the acquired network security data;
dividing the acquired network security data into a plurality of data blocks by adopting a data dividing module;
step 3, preprocessing the network security data after division;
the method comprises the steps that a data processing module is adopted to preprocess divided network security data, the data processing module comprises a main controller, a data conversion unit, a data cleaning unit and a data sorting unit, the main controller is used for adjusting the working states of all modules of the situation data processing system, the data conversion unit is used for converting network security situation information into a machine receiving form through a neural network converter, the data cleaning unit comprises a filling subunit and a denoising subunit, the filling subunit fills up a data incomplete part through an interpolation algorithm, the denoising subunit is used for removing redundant parts of data through a wavelet transformation algorithm, the data sorting unit is used for sorting the cleaned data into a sequence according to time, the output end of the data conversion unit is connected with the input end of the data cleaning unit, and the output end of the data cleaning unit is connected with the input end of the data sorting unit;
step 4, classifying and storing the ordered network security data;
classifying the network security data by adopting a data classification module, wherein the data classification module comprises a discrimination unit and a storage unit, the discrimination unit is used for discriminating the network security data according to time and situation by a clustering and mixing algorithm, the storage unit is used for carrying out distributed storage on each type of network security data, and the output end of the discrimination unit is connected with the input end of the storage unit;
the working method of the clustering mixing algorithm comprises the following steps: the method comprises the steps of firstly carrying out dimension reduction and mapping on high-dimension data by using a self-organizing map algorithm, obtaining a self-organizing map, then taking nodes on the self-organizing map as initial clusters, then carrying out K-means clustering algorithm on each cluster to divide the clusters into sub-clusters, then calculating the distance between each sub-cluster and other sub-clusters as well as the initial clusters, finally merging the sub-clusters by adopting split hierarchical clusters until the number of the clusters reaches saturation, then carrying out K-means algorithm on all the clusters to obtain mass centers, and representing the clusters by the mass centers.
The working principle of the clustering mixing algorithm is as follows: the data points in the data set are divided according to the distance between the data points and the cluster center, so that the distances between the data points in the same cluster are relatively close, the distances between the data points in different clusters are relatively far, the hyperplane capable of best separating different types of data is selected, the distances between the data points in different types are maximized, a high-level and large-scale data set can be effectively processed, meanwhile, the calculation complexity is reduced, a more accurate cluster center can be obtained by adopting a self-organizing map algorithm, the data distribution situation can be better represented, the calculation complexity can be effectively reduced when the large-scale data is processed by adopting a split hierarchical cluster algorithm, the algorithm efficiency is improved, a relatively accurate cluster result can be obtained by adopting a K-means cluster algorithm, the high-dimensional characteristics can be processed by adopting a self-organizing map algorithm, and meanwhile, the defect that the K-means algorithm is susceptible to the initial cluster center can be avoided, as shown in the table 1.
TABLE 1
Grade | Center point | Category(s) | Results |
1 | (1,0,0,0,0) | A | 1 |
2 | (0,1,0,0,0) | C | 3 |
3 | (0,0,1,0,0) | B | 2 |
4 | (0,0,0,1,0) | D | 4 |
As can be seen from table 1, the network security data information is divided into 4 categories, which are respectively: a is static data, B is dynamic data, C is real-time data, D is graph data, and the graph data is outwards diffused according to different center points, wherein the network data range included in the farthest distance is the same type of data;
step 5, carrying out data error detection and recovery on the classified network security data by adopting a corresponding processing mode;
the method comprises the steps that error detection and recovery of network safety data are carried out through a detection recovery module, the detection recovery module comprises a detection unit, an analysis unit, a screening unit, a scheduling unit and a recovery unit, the detection unit is used for carrying out abnormal detection on classified network safety data through an abnormal threat model, the analysis unit is used for discovering abnormal reasons of detection, the scheduling unit is used for automatically distributing and scheduling computing nodes through a scheduling algorithm to detect abnormal threats, the screening unit is used for monitoring each computing node through a search engine and isolating the abnormal computing nodes, the recovery unit is used for recovering the computing nodes monitored for abnormality through restarting and self-checking, the output end of the detection unit is connected with the input end of the analysis unit, the output end of the analysis unit is connected with the input end of the scheduling unit, the output end of the scheduling unit is connected with the input end of the screening unit, and the output end of the screening unit is connected with the input end of the recovery unit;
step 6, carrying out feedback evaluation and visualization on the network security situation data processing result and the computing node monitoring result;
the intelligent display module is arranged to visually display network security situation data processing results, computing node monitoring results and evaluation feedback results thereof and generate corresponding graphic reports, the intelligent display module comprises a display unit, an evaluation unit and a wireless transmission unit, the display unit displays the data processing results and the computing node monitoring results through a touch screen, the evaluation unit evaluates and feeds back the data processing results and the computing node monitoring results through an evaluation matching algorithm, the wireless transmission unit performs wireless communication with a plurality of terminals through a TCP/IP protocol, the output end of the evaluation unit is connected with the input end of the display unit, and the output end of the display unit is connected with the input end of the wireless transmission unit.
The output end of the main controller is respectively connected with the input ends of the data collection module, the data division module, the data processing module, the data classification module, the detection recovery module and the intelligent display module, the output end of the data collection module is connected with the input end of the data division module, the output end of the data division module is connected with the input end of the data processing module, the output end of the data processing module is connected with the input end of the data classification module, the output end of the data classification module is connected with the input end of the detection recovery module, and the output end of the detection recovery module is connected with the input end of the intelligent display module.
The neural network converter further comprises a preceding neuron, a preset information conversion algorithm, a preset pulse conversion algorithm and a following neuron, wherein the preceding neuron receives network security situation information and adopts a preset information conversion method to linearly convert the network security situation information into coded pulse input information, and then the coded pulse input information is converted into neuron information through the preset pulse conversion method in a nonlinear manner and the following neuron outputs data.
The working principle of the neural network converter is as follows: the preceding neuron refers to a neuron that receives input information, performs weighted and nonlinear transformation on the input information, and then transfers the transformed result to other neurons, and the preceding neuron generally includes an input membrane potential, a threshold voltage, and a membrane potential recovery rate. The preset information conversion algorithm refers to a sensor algorithm and an adaptive differential evolution algorithm for converting input information into an internal state of a neuron. These algorithms mainly utilize nonlinear transformation and feedback mechanisms inside neurons to convert input information into state variables such as membrane potential inside neurons. The preset pulse conversion algorithm is used for converting the internal states of the neurons into a threshold synaptic algorithm and a pulse integration algorithm of an output pulse sequence. These algorithms can produce output pulse trains of different times, amplitudes and frequencies, depending on different combinations of internal states. The subsequent neurons refer to neurons that receive the output pulses, which may receive multiple input pulses, then weight and non-linearly transform the pulses, and pass on to the next layer of neurons. Subsequent neurons typically include output membrane potentials and response time constants. Extracting features of the network information by using a pre-trained convolutional neural network and generating an attention mask indicating a target position profile, the attention mask characterizing a probability that the corresponding element belongs to a target class; the attention mask is used as prior information to guide a converter network to pay attention to the identification area to determine the network information category, and the neural network converter is obtained by training with the set loss function as an optimization target.
Further, the main controller comprises an FPGA+DSP processing module, the DSP processing module is an acquisition chip of ATMega328 model, the DSP processing module integrates a 14-path GPIO interface, a 6-path PWM interface, a 12-bit ADC interface, a UART serial port, a 1-path SPI interface and a 1-path I2C interface, and the FPGA processing module is a Spartm-7 series XC7S15-2CSGA225I chip.
The working principle of the main controller is as follows: the method comprises the steps that a main controller firstly controls a data collection module to receive network security situation data information, then the received network security situation data information is divided into a plurality of data blocks through a data division module, then a data processing module is controlled to convert, clean and sort the divided data blocks, a classification module is controlled to divide the network security data into 4 types, then threat detection is carried out on the classified network data and computing nodes through a detection recovery module, and finally a graph is visually displayed on the network security situation data processing result and the computing node monitoring result through an intelligent display module, and the judgment feedback result of the network security situation data processing result and the computing node monitoring result are judged.
Further, the abnormal threat model comprises a loading unit, a calculating unit, a learning unit and an identification unit, wherein the loading unit reads data from a storage unit through a language describing a data stream and translates the data stream into data to be executed, the calculating unit adjusts large batches of data to be executed in real time according to weight values and threshold values, nonlinear transformation is carried out on calculation results through an activation function to extract complex features, the learning unit continuously adjusts weight and threshold values in a neural network according to a preset objective function or an error function to iteratively train the processed execution data and adjust a basic network model, the identification unit predicts abnormal threat types of classified data by adopting different activation functions and objective functions, the output end of the loading unit is connected with the input end of the calculating unit, the output end of the calculating unit is connected with the input end of the learning unit, and the output end of the learning unit is connected with the input end of the identification unit.
The working principle of the abnormal threat model is as follows: the classified network security data is iteratively trained to form a normal data network model, and then the model is utilized to further resolve real-time data and computing nodes so as to find the specific position of the outlier.
Further, the working method of the computing unit is as follows: firstly, verifying the format of a real-time data stream, dividing stream data into a plurality of data blocks according to the length, the position and the batch size of the data, distributing the data blocks to different computing nodes for execution, loading and converting the data into elastic distributed key value pairs suitable for mapping task reading, matching according to similar characteristics between a source domain and a target domain, outputting a series of key words and key value pairs as intermediate results, partitioning, sorting, merging and merging the key words and the key values, then delivering the key words and the key value pairs to corresponding simplification tasks for parallel processing on a plurality of machines, and summarizing and calculating a key value list with the same key words by the same simplification task, thereby executing logic output and storing the results.
Further, the learning unit performs feature transformation by adopting a convolution combination of 3×3 and 5×5, performs feature downsampling by adopting a maximum pooling layer and performs feature processing by adopting 1×1 convolution, performs classification processing by adopting three nonlinear layers, and finally improves training performance by adopting a combined activation function, wherein the combined activation function is as follows:
in the formula (1), n is the stacking number, a i B for scaling parameters i A (x) is a combined activation function, x is input data, lambda is a super parameter,
the working process of the learning unit is as follows: the input data information is subjected to feature transformation through convolution of 3×3 and 5×5, then the data information subjected to feature transformation is subjected to feature downsampling by adopting a maximum pooling layer, then is subjected to feature processing by adopting convolution of 1×1, and finally the processed data information is classified by three nonlinear layers, as shown in table 2.
Table 2 class learning table
Input type | Feature transformation | Feature processing | Effects/s |
Static data | 128×128 | 32×32 | 1.7 |
Dynamic data | 256×256 | 64×64 | 4.6 |
Graph data | 1024×1024 | 128×128 | 8.9 |
As can be seen from table 2, the image size finally obtained is different for different types of data by the same processing manner, so the recognition speed is also different, the higher the depth is, the more time is consumed for data processing, but the model is consistent in terms of processing accuracy.
Further, the working method of the loading unit is as follows: firstly, receiving a request, acquiring data from a storage unit, then analyzing and processing the acquired data, translating a data stream into executable data, performing code conversion and character set processing on text data, performing byte order conversion and data type conversion on binary data, and finally taking the translated data as the data to be executed.
While specific embodiments of the present invention have been described above, it will be understood by those skilled in the art that these specific embodiments are by way of example only, and that various omissions, substitutions, and changes in the form and details of the methods and systems described above may be made by those skilled in the art without departing from the spirit and scope of the invention. For example, it is within the scope of the present invention to combine the above-described method steps to perform substantially the same function in substantially the same way to achieve substantially the same result. Accordingly, the scope of the invention is limited only by the following claims.
Claims (7)
1. A situation data information processing method based on computer network security is characterized in that: the method comprises the following steps:
step 1, acquiring network security situation data of various types;
acquiring multiple types of network security data through a data acquisition module, wherein the data acquisition module comprises a content acquisition unit and a link filtering unit, the content acquisition unit is used for acquiring multiple types of network security data content, and the link filtering unit is used for removing network links of the acquired network security data;
step 2, dividing the acquired network security data;
dividing the acquired network security data into a plurality of data blocks by adopting a data dividing module;
step 3, preprocessing the network security data after division;
the method comprises the steps that a data processing module is adopted to preprocess divided network security data, the data processing module comprises a main controller, a data conversion unit, a data cleaning unit and a data sorting unit, the main controller is used for adjusting the working states of all modules of the situation data processing system, the data conversion unit is used for converting network security situation information into a machine receiving form through a neural network converter, the data cleaning unit comprises a filling subunit and a denoising subunit, the filling subunit fills up a data incomplete part through an interpolation algorithm, the denoising subunit is used for removing redundant parts of data through a wavelet transformation algorithm, the data sorting unit is used for sorting the cleaned data into a sequence according to time, the output end of the data conversion unit is connected with the input end of the data cleaning unit, and the output end of the data cleaning unit is connected with the input end of the data sorting unit;
step 4, classifying and storing the ordered network security data;
classifying the network security data by adopting a data classification module, wherein the data classification module comprises a discrimination unit and a storage unit, the discrimination unit is used for discriminating the network security data according to time and situation by a clustering and mixing algorithm, the storage unit is used for carrying out distributed storage on each type of network security data, and the output end of the discrimination unit is connected with the input end of the storage unit;
the working method of the clustering mixing algorithm comprises the following steps: firstly, performing dimension reduction and mapping on high-dimension data by using a self-organizing map algorithm to obtain a self-organizing map, then taking nodes on the self-organizing map as initial clusters, then executing a K-means clustering algorithm on each cluster to divide the clusters into sub-clusters, then calculating the distance between each sub-cluster and other sub-clusters and the initial clusters, finally merging the sub-clusters by adopting split hierarchical clusters until the number of the clusters reaches saturation, then executing a K-means algorithm on all the clusters to obtain mass centers, and representing the clusters by the mass centers;
step 5, carrying out data error detection and recovery on the classified network security data by adopting a corresponding processing mode;
the method comprises the steps that error detection and recovery of network safety data are carried out through a detection recovery module, the detection recovery module comprises a detection unit, an analysis unit, a screening unit, a scheduling unit and a recovery unit, the detection unit is used for carrying out abnormal detection on classified network safety data through an abnormal threat model, the analysis unit is used for discovering abnormal reasons of detection, the scheduling unit is used for automatically distributing and scheduling computing nodes through a scheduling algorithm to detect abnormal threats, the screening unit is used for monitoring each computing node through a search engine and isolating the abnormal computing nodes, the recovery unit is used for recovering the computing nodes monitored for abnormality through restarting and self-checking, the output end of the detection unit is connected with the input end of the analysis unit, the output end of the analysis unit is connected with the input end of the scheduling unit, the output end of the scheduling unit is connected with the input end of the screening unit, and the output end of the screening unit is connected with the input end of the recovery unit;
step 6, carrying out feedback evaluation and visualization on the network security situation data processing result and the computing node monitoring result;
the intelligent display module is arranged to visually display network security situation data processing results, computing node monitoring results and evaluation feedback results thereof and generate corresponding graphic reports, the intelligent display module comprises a display unit, an evaluation unit and a wireless transmission unit, the display unit displays the data processing results and the computing node monitoring results through a touch screen, the evaluation unit evaluates and feeds back the data processing results and the computing node monitoring results through an evaluation matching algorithm, the wireless transmission unit performs wireless communication with a plurality of terminals through a TCP/IP protocol, the output end of the evaluation unit is connected with the input end of the display unit, and the output end of the display unit is connected with the input end of the wireless transmission unit.
2. A situation data information processing method based on computer network security as claimed in claim 1, wherein: the neural network converter comprises a preceding neuron, a preset information conversion algorithm, a preset pulse conversion algorithm and a following neuron, wherein the preceding neuron receives network security situation information and adopts a preset information conversion method to linearly convert the network security situation information into coded pulse input information, and then the coded pulse input information is converted into neuron information through the preset pulse conversion method in a nonlinear manner and the following neuron outputs data.
3. A situation data information processing method based on computer network security as claimed in claim 1, wherein: the main controller comprises an FPGA+DSP processing module, the DSP processing module is an acquisition chip of ATMega328 model, the DSP processing module integrates a 14-path GPIO interface, a 6-path PWM interface, a 12-bit ADC interface, a UART serial port, a 1-path SPI interface and a 1-path I2C interface, and the FPGA processing module is a Spartan-7 series XC7S15-2CSGA225I chip.
4. A situation data information processing method based on computer network security as claimed in claim 1, wherein: the abnormal threat model comprises a loading unit, a calculating unit, a learning unit and an identification unit, wherein the loading unit reads data from a storage unit through a language describing a data stream and translates the data stream into data to be executed, the calculating unit adjusts large batches of data to be executed in real time according to weight values and threshold values, nonlinear transformation is carried out on calculation results through an activation function to extract complex features, the learning unit continuously adjusts weight and threshold values in a neural network according to a preset objective function or an error function to iteratively train the processed execution data and adjust a basic network model, the identification unit predicts abnormal threat types of classified data by adopting different activation functions and objective functions, the output end of the loading unit is connected with the input end of the calculating unit, the output end of the calculating unit is connected with the input end of the learning unit, and the output end of the learning unit is connected with the input end of the identification unit.
5. The situation data information processing method based on computer network security according to claim 4, wherein: the working method of the computing unit comprises the following steps: firstly, verifying the format of a real-time data stream, dividing stream data into a plurality of data blocks according to the length, the position and the batch size of the data, distributing the data blocks to different computing nodes for execution, loading and converting the data into elastic distributed key value pairs suitable for mapping task reading, matching according to similar characteristics between a source domain and a target domain, outputting a series of key words and key value pairs as intermediate results, partitioning, sorting, merging and merging the key words and the key values, then delivering the key words and the key value pairs to corresponding simplification tasks for parallel processing on a plurality of machines, and summarizing and calculating a key value list with the same key words by the same simplification task, thereby executing logic output and storing the results.
6. The situation data information processing method based on computer network security according to claim 4, wherein: the learning unit adopts convolution of 3×3 and 5×5 to perform feature transformation, adopts a maximum pooling layer to perform feature downsampling and 1×1 convolution to perform feature processing, adopts three nonlinear layers to perform classification processing, and finally improves training performance by a combined activation function, wherein the combined activation function is as follows:
in the formula (1), n is the stacking number, a i B for scaling parameters i A (x) is an original activation function, x is input data, and lambda is a super parameter.
7. The situation data information processing method based on computer network security according to claim 4, wherein: the working method of the loading unit comprises the following steps: firstly, receiving a request, acquiring data from a storage unit, then analyzing and processing the acquired data, translating a data stream into executable data, performing code conversion and character set processing on text data, performing byte order conversion and data type conversion on binary data, and finally taking the translated data as the data to be executed.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311019142.8A CN116756225B (en) | 2023-08-14 | 2023-08-14 | Situation data information processing method based on computer network security |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311019142.8A CN116756225B (en) | 2023-08-14 | 2023-08-14 | Situation data information processing method based on computer network security |
Publications (2)
Publication Number | Publication Date |
---|---|
CN116756225A CN116756225A (en) | 2023-09-15 |
CN116756225B true CN116756225B (en) | 2023-11-07 |
Family
ID=87951753
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311019142.8A Active CN116756225B (en) | 2023-08-14 | 2023-08-14 | Situation data information processing method based on computer network security |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116756225B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117309824B (en) * | 2023-11-08 | 2024-03-26 | 广州市市维检测有限公司 | Photocatalyst coating layer detection system and method |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105681303A (en) * | 2016-01-15 | 2016-06-15 | 中国科学院计算机网络信息中心 | Big data driven network security situation monitoring and visualization method |
WO2018081742A1 (en) * | 2016-10-31 | 2018-05-03 | Acentium Inc. | Methods and systems for ranking, filtering and patching detected vulnerabilities in a networked system |
CN209085657U (en) * | 2017-08-02 | 2019-07-09 | 强力物联网投资组合2016有限公司 | For data gathering system related or industrial environment with chemical production technology |
CN110392048A (en) * | 2019-07-04 | 2019-10-29 | 湖北央中巨石信息技术有限公司 | Network security situation awareness model and method based on CE-RBF |
US10673880B1 (en) * | 2016-09-26 | 2020-06-02 | Splunk Inc. | Anomaly detection to identify security threats |
CN112651006A (en) * | 2020-12-07 | 2021-04-13 | 中国电力科学研究院有限公司 | Power grid security situation perception platform framework |
CN112703457A (en) * | 2018-05-07 | 2021-04-23 | 强力物联网投资组合2016有限公司 | Method and system for data collection, learning and machine signal streaming for analysis and maintenance using industrial internet of things |
CN113067728A (en) * | 2021-03-17 | 2021-07-02 | 中国人民解放军海军工程大学 | Network security attack and defense test platform |
CN115481673A (en) * | 2021-06-14 | 2022-12-16 | 雷德本德有限公司 | Enhancing vehicle network security using staged machine learning |
CN115484175A (en) * | 2022-10-27 | 2022-12-16 | 北京六方云信息技术有限公司 | Intelligent manufacturing network attack and defense display method, device and system and storage medium |
CN116366277A (en) * | 2022-12-07 | 2023-06-30 | 国网新疆电力有限公司信息通信公司 | Network security situation assessment method for information fusion |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110214157A1 (en) * | 2000-09-25 | 2011-09-01 | Yevgeny Korsunsky | Securing a network with data flow processing |
US20070192863A1 (en) * | 2005-07-01 | 2007-08-16 | Harsh Kapoor | Systems and methods for processing data flows |
US11112784B2 (en) * | 2016-05-09 | 2021-09-07 | Strong Force Iot Portfolio 2016, Llc | Methods and systems for communications in an industrial internet of things data collection environment with large data sets |
-
2023
- 2023-08-14 CN CN202311019142.8A patent/CN116756225B/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105681303A (en) * | 2016-01-15 | 2016-06-15 | 中国科学院计算机网络信息中心 | Big data driven network security situation monitoring and visualization method |
US10673880B1 (en) * | 2016-09-26 | 2020-06-02 | Splunk Inc. | Anomaly detection to identify security threats |
WO2018081742A1 (en) * | 2016-10-31 | 2018-05-03 | Acentium Inc. | Methods and systems for ranking, filtering and patching detected vulnerabilities in a networked system |
CN209085657U (en) * | 2017-08-02 | 2019-07-09 | 强力物联网投资组合2016有限公司 | For data gathering system related or industrial environment with chemical production technology |
CN110073301A (en) * | 2017-08-02 | 2019-07-30 | 强力物联网投资组合2016有限公司 | The detection method and system under data collection environment in industrial Internet of Things with large data sets |
CN112703457A (en) * | 2018-05-07 | 2021-04-23 | 强力物联网投资组合2016有限公司 | Method and system for data collection, learning and machine signal streaming for analysis and maintenance using industrial internet of things |
CN110392048A (en) * | 2019-07-04 | 2019-10-29 | 湖北央中巨石信息技术有限公司 | Network security situation awareness model and method based on CE-RBF |
CN112651006A (en) * | 2020-12-07 | 2021-04-13 | 中国电力科学研究院有限公司 | Power grid security situation perception platform framework |
CN113067728A (en) * | 2021-03-17 | 2021-07-02 | 中国人民解放军海军工程大学 | Network security attack and defense test platform |
CN115481673A (en) * | 2021-06-14 | 2022-12-16 | 雷德本德有限公司 | Enhancing vehicle network security using staged machine learning |
CN115484175A (en) * | 2022-10-27 | 2022-12-16 | 北京六方云信息技术有限公司 | Intelligent manufacturing network attack and defense display method, device and system and storage medium |
CN116366277A (en) * | 2022-12-07 | 2023-06-30 | 国网新疆电力有限公司信息通信公司 | Network security situation assessment method for information fusion |
Non-Patent Citations (3)
Title |
---|
Review of Power Spatio-Temporal Big Data Technologies for Mobile Computing in Smart Grid;Y. Ma 等;《in IEEE Access》;第7卷;174612-174628 * |
工业互联网的安全挑战及应对策略;陶耀东 等;《中兴通讯技术》;第22卷(第5期);36-41+46 * |
数据驱动的物联网安全威胁检测与建模;杨威超;《中国优秀硕士学位论文全文数据库信息科技辑》(第(2020)02期);I136-501 * |
Also Published As
Publication number | Publication date |
---|---|
CN116756225A (en) | 2023-09-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109639739B (en) | Abnormal flow detection method based on automatic encoder network | |
WO2023044978A1 (en) | Adversarial-flow-model-based unsupervised fault diagnosis method for mechanical device | |
CN111832647A (en) | Abnormal flow detection system and method | |
Du et al. | GAN-based anomaly detection for multivariate time series using polluted training set | |
CN116756225B (en) | Situation data information processing method based on computer network security | |
CN109639734B (en) | Abnormal flow detection method with computing resource adaptivity | |
CN112131907A (en) | Method and device for training classification model | |
CN112367303B (en) | Distributed self-learning abnormal flow collaborative detection method and system | |
CN110851654A (en) | Industrial equipment fault detection and classification method based on tensor data dimension reduction | |
CN112884121A (en) | Traffic identification method based on generation of confrontation deep convolutional network | |
CN114416423B (en) | Root cause positioning method and system based on machine learning | |
CN113609480B (en) | Multipath learning intrusion detection method based on large-scale network flow | |
CN114548295A (en) | Bearing fault classification system and method based on multi-scale domain adaptive network | |
CN112926269B (en) | Method and system for grouping and cleaning power plant edge node data | |
Li et al. | Class imbalanced fault diagnosis via combining K-means clustering algorithm with generative adversarial networks | |
CN113723592A (en) | Fault diagnosis method based on wind power gear box monitoring system | |
CN117743933A (en) | Method and device for determining invalid alarm information, storage medium and electronic device | |
Wang | Research on the fault diagnosis of mechanical equipment vibration system based on expert system | |
Wang et al. | Knowledge and Data Dual-Driven Fault Diagnosis in Industrial Scenarios: A Survey | |
CN116318925A (en) | Multi-CNN fusion intrusion detection method, system, medium, equipment and terminal | |
CN116032790A (en) | Method, device and system for identifying, diagnosing and predicting massive data flow anomalies of dispatching automation system | |
CN114358058B (en) | Wireless communication signal open set identification method and system based on deep neural network | |
CN112991093B (en) | Electric larceny detection method and system based on edge calculation | |
CN108958221A (en) | Equipment fault diagnosis method based on industrial Internet of Things Yu support vector machines multi-classification algorithm | |
Hao et al. | New fusion features convolutional neural network with high generalization ability on rolling bearing fault diagnosis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right |
Effective date of registration: 20231124 Address after: 134 Huaihai South Road, Huai'an City, Jiangsu Province Patentee after: HUAI'AN POWER SUPPLY BRANCH OF STATE GRID JIANGSU ELECTRIC POWER Co.,Ltd. Address before: Room 018, 4th Floor, Building 6, Fengxin Road, Yuhuatai District, Nanjing City, Jiangsu Province, 210000 Patentee before: Nanjing zhanyan Information Technology Co.,Ltd. |
|
TR01 | Transfer of patent right |