CN117354251A - Automatic extraction method for electric power Internet of things terminal characteristics - Google Patents

Abstract

The invention discloses an automatic extraction method of electric power Internet of things terminal characteristics, which comprises the following steps: identifying flow information of the electric power internet of things terminal; setting a feature classification rule, classifying and labeling the equipment feature information, and dividing the equipment feature information into shallow features and depth features; shallow layer features of the equipment are extracted at the edge nodes, and depth features of the equipment are extracted at the cloud; analyzing the abnormal characteristics of each label, performing correlation analysis on the extracted characteristics, and screening out characteristic values with high correlation; constructing a feature extraction model of multiple modes; and combining workflow automation arrangement to construct a feature extraction model with multiple modes. According to the method and the device, on the basis of analyzing the message, the flow uploaded by the device is detected and graded, different characteristic information extraction is executed at the edge terminal and the cloud according to actual service conditions, and the characteristic extraction process is automated, so that the calculation efficiency can be improved, the influence of network time delay is reduced, and the cloud calculation and storage cost is reduced.

Description

Automatic extraction method for electric power Internet of things terminal characteristics

Technical Field

The invention relates to the technical field of electric power internet of things terminal equipment feature extraction, in particular to an automatic extraction method of electric power internet of things terminal features.

Background

With the vigorous development of power grid infrastructure, the construction of the electric power Internet of things is taken as an important channel and means for further implementing the national 'new infrastructure'. Under the current situation, the electric power internet of things is facing massive data acquisition, high-performance equipment access and other requirements, and has higher requirements on data convergence and access capacity of a system platform.

In the electric power Internet of things, functions and configuration principles of an Internet of things management platform and an edge Internet of things terminal are different, requirements on power grid equipment, client side sensing, acquisition, monitoring and the like are increasingly urgent, how to timely sense and acquire mass operation information of various equipment and client sides, unified Internet of things management and terminal standardized access are achieved, ecology shared by data sharing is formed, data asset value is fully exerted, expandability and consistency of equipment capability expansion are achieved, and work efficiency can be greatly improved.

The current internet of things management platform technology can realize the effective management of parts of edge equipment, but in the aspect of terminal perception, the related system, standard and basic technology of terminal identification are relatively lacking, and the automatic registration and running state perception of the terminal are difficult to realize. Because of various types of sensing terminals, lack of a unified terminal identification mechanism and the like, the types of access terminals, transmission protocols and information data types are required to be manually configured, the characteristics of the terminal equipment cannot be automatically acquired, the access efficiency of the terminal is low, the automatic sensing of the running state and other information of the opposite terminal equipment cannot be realized, and the efficient lean management and control requirements of an electric power system on the opposite terminal equipment are difficult to realize.

The invention of patent publication No. CN1 15801411A discloses a high-order data feature extraction and identification method for electric power Internet of things attack behaviors, which is mainly based on secondary data construction of side channel information of electric power Internet of things terminal equipment, mainly realizes programmed extraction of attack behavior features, and is based on a simplified electric power Internet of things abnormal behavior primary data feature analysis and extraction model for practical high-order data feature analysis and extraction. The invention of patent publication number CN114492613A discloses a method, a system, a terminal and a readable storage medium for identifying equipment of the Internet of things and non-Internet of things, and the invention relates to feature extraction of the terminal of the Internet of things, which is characterized in that flow features and protocol features are extracted from network flow so as to establish an initial random forest model; and extracting the characteristics of the newly added equipment, selecting the representative equipment from the newly added equipment for marking verification, and using the representative equipment for model updating, judging the importance of the model based on characteristic reduction in the model updating process, deleting the unimportant characteristics, and improving the model identification precision. Both of these patents do not allow for automated extraction of characteristics of the internet of things terminal.

Disclosure of Invention

The invention discloses an automatic extraction method of characteristics of an electric power Internet of things terminal, which is used for detecting and grading the flow uploaded by equipment on the basis of analyzing a message, executing different characteristic information extraction on an edge terminal and a cloud as required according to the actual service condition, and automatizing the flow of the characteristic extraction, so that the calculation efficiency can be improved, the influence of network time delay can be reduced, and the cloud calculation and storage cost can be reduced.

In order to achieve the technical purpose, the invention adopts the following technical scheme:

an automatic extraction method of electric power internet of things terminal features comprises the following steps:

s1, identifying flow information of an electric power internet of things terminal by adopting a depth detection technology, and storing an equipment signature obtained by identification into a corresponding signature database through hash mapping;

s2, setting a feature classification rule, classifying and labeling the equipment feature information, and dividing the equipment feature information into shallow features and depth features; aiming at the flow information of the electric power internet of things terminal, shallow layer characteristics of equipment are extracted at edge nodes, and depth characteristics of the equipment are extracted at a cloud;

s3, analyzing the abnormal characteristics of each label, performing correlation analysis on the characteristics extracted in the step S2, and screening out characteristic values with correlation higher than a correlation threshold; performing dimension reduction processing on the screened characteristic values, clustering or classifying the characteristic values after the dimension reduction processing, and constructing a characteristic extraction model with multiple modes;

and S4, combining the workflow automation programming step S3 to construct a multi-mode feature extraction model to finish the automatic extraction of the flow features.

Further, in step S1, the process of identifying the flow information of the electric power internet of things terminal by adopting the depth detection technology and storing the identified device signature into the corresponding signature database through the hash mapping includes the following steps:

analyzing the flow information of the captured electric power internet of things terminal, and extracting key information including a source IP address, a target IP address, a port number and a protocol type;

carrying out local segmentation on flow information of the electric power Internet of things terminal obtained through analysis, carrying out hash calculation on all fragments, generating a hash value abstract with fixed length, and storing the generated hash value abstract in a signature database; the similarity of the flow information of the electric power internet of things terminal is evaluated by comparing the similarity of the two abstracts;

when a new electric power internet of things terminal joins the network, according to Nilsima hash of the network flow computing device, and comparing the Nilsima hash with the hash value abstract stored in the signature database, setting a device type label with the highest average hash similarity score for the new electric power internet of things terminal.

Further, for each electric power internet of things terminal, a Nilsima hash is generated for the network communication flow of the electric power internet of things terminal, and the generated Nilsima hash is stored in a signature database.

Further, in step S2, a feature classification rule is set, and the device feature information is classified and labeled, and is divided into shallow features and depth features; aiming at the flow information of the electric power internet of things terminal, shallow layer characteristics of equipment are extracted at edge nodes, and depth characteristics of the equipment are extracted at a cloud end, wherein the method comprises the following steps:

s21, selecting a representative electric power internet of things terminal device, collecting characteristic data of all selected devices within a certain time period, and constructing to obtain a characteristic data set;

s22, designing a feature representation mode according to the feature essential attribute information;

s23, performing unsupervised learning on the feature data set by adopting a deep learning algorithm, and automatically learning a feature space structure;

s24, constructing a preliminary static grading rule based on the learning result of the step S23 to obtain a rule model;

s25, reporting the extracted new features and the evaluation results in real time by the edge node to serve as a group of samples;

s26, periodically collecting samples by the cloud, retraining the rule model together with the historical sample data, dynamically adjusting the rule model according to the training result, and modifying the classification result of part of the characteristics.

Further, in step S2, in step S26, the rule model is dynamically adjusted in combination with the device running state, the cloud running state and the training result.

Further, in step S3, the abnormal characteristics of each label are analyzed, the correlation analysis is performed on the features extracted in step S2 through a statistical algorithm or a feature selection algorithm of variance analysis, and feature values with correlation higher than a correlation threshold are screened out; and then adopting principal component analysis to perform dimension reduction treatment on the screened characteristic values.

Further, in step S3, clustering is performed based on the distance between the data points using KNN clustering algorithm, and similar data points are grouped together; the new data is classified using a BP neural network or decision tree algorithm.

Further, in step S4, the process of automatically extracting the flow characteristics by combining the workflow automation programming step S3 to construct the obtained feature extraction model with multiple modes includes the following steps:

s41, serializing the feature extraction model after training and optimization into a model object, and independently extracting the feature extraction model from a training environment;

s42, integrating the derived feature extraction model into independent service, integrating the independent service into an automation system, and providing an API interface for other application programs to call;

s43, installing an agent or a sensor on the electric power internet of things terminal equipment, and transmitting the data stream to an automation system by using a network protocol;

s44, defining a workflow and task dependency relationship of a feature extraction application by using an automatic flow management tool, automatically triggering a feature extraction task according to actual business conditions, extracting features by adopting a feature extraction model, and managing the execution state of the extraction task in real time;

s45, outputting the extracted characteristic result to a specified database for storage;

s46, judging whether an abnormal situation occurs or not by monitoring the execution state, the performance index and the log of the feature extraction task, and triggering an alarm notification if the abnormal situation occurs.

Further, the automated extraction method further comprises: analyzing the characteristic results in the appointed database, and feeding back the analysis results to the corresponding electric power internet of things terminal in real time, so that the electric power internet of things terminal carries out self-adaptive modulation according to the characteristic results; the method specifically comprises the following substeps:

extracting various operation characteristics of the electric power Internet of things terminal in real time;

the extracted characteristic results are sent to the equipment management system in real time through a communication network or a message queue and the like; wherein, when extracting the result, a type mark is added to each feature record, and the type mark comprises: performance class features, error class features, usage pattern class features, deployment class features, and environment class features;

after receiving the characteristic results, the equipment management system analyzes the message body through reverse serialization, obtains the characteristic results and the type marks thereof, routes the characteristic results, distributes the characteristic results to subsystem interfaces of corresponding equipment according to different characteristic types, and each subsystem automatically executes adjustment actions according to the characteristic results, so that the electric power Internet of things terminal adopts an automatic configuration mode and automatically optimizes by applying the adjustment results in real time;

the log system re-collects the later data, observes the adjustment effect, and loops the feature extraction and optimization process.

Further, the log system re-collects the later data, observes the adjustment effect, and the process of the loop feature extraction and optimization process includes the following steps:

the log collecting system re-collects the equipment operation log according to a certain time sequence, compares the collected equipment operation log with the original log, and extracts a newly added log and a modified log;

the extraction system updates the original model in real time according to the newly added data, and extracts newly added characteristic indexes; comparing the newly added characteristic index with the historical characteristic, and evaluating whether the adjusted newly added characteristic index tends to an expected effect; the optimization degree of key indexes is obtained through characteristic effect monitoring and log analysis, and monitoring results are fed back to the equipment management system;

the equipment management system evaluates whether the effect of the last round of adjustment strategy reaches the expected purpose according to the monitoring result; if the adjustment strategy is not achieved, a new optimization scheme is generated through the machine learning lifting model, and a new round of adjustment strategy is issued to the electric power Internet of things terminal for implementation by the equipment management system.

Compared with the prior art, the invention has the following beneficial effects:

according to the automatic extraction method for the characteristics of the electric power internet of things terminal, provided by the invention, on the basis of analyzing the message, the flow uploaded by the equipment is detected and classified, different characteristic information extraction is executed at the edge terminal and the cloud terminal according to the actual service conditions, and the characteristic extraction flow is automated, so that the calculation efficiency can be improved, the influence of network time delay is reduced, and the cloud computing and storage cost is reduced.

Drawings

FIG. 1 is a flow chart of an automated extraction method of electric power Internet of things terminal features of the present invention;

FIG. 2 is a schematic diagram of a flow data detection flow of a device;

fig. 3 is a schematic diagram of a feature extraction process.

Detailed Description

Embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

Referring to fig. 1, the embodiment of the invention discloses an automatic extraction method of electric power internet of things terminal characteristics, which comprises the following steps:

s1, identifying flow information of an electric power internet of things terminal by adopting a depth detection technology, and storing an equipment signature obtained by identification into a corresponding signature database through hash mapping;

s2, setting a feature classification rule, classifying and labeling the equipment feature information, and dividing the equipment feature information into shallow features and depth features; aiming at the flow information of the electric power internet of things terminal, shallow layer characteristics of equipment are extracted at edge nodes, and depth characteristics of the equipment are extracted at a cloud;

s3, analyzing the abnormal characteristics of each label, performing correlation analysis on the characteristics extracted in the step S2, and screening out characteristic values with correlation higher than a correlation threshold; performing dimension reduction processing on the screened characteristic values, clustering or classifying the characteristic values after the dimension reduction processing, and constructing a characteristic extraction model with multiple modes;

and S4, combining the workflow automation programming step S3 to construct a multi-mode feature extraction model to finish the automatic extraction of the flow features.

The embodiment of the invention relates to an automatic extraction method of electric power Internet of things terminal characteristics, which has the premise that corresponding message information can be fully extracted through deep analysis of a protocol. The overall flow of the automatic extraction method is that the flow information of the equipment is fully identified by adopting a depth detection technology and is saved through hash mapping. And then, respectively processing different types of characteristic information at the local and cloud sides by using a characteristic grading extraction method, and automatically arranging by combining a plurality of algorithm models and workflow, so as to finish the automatic extraction of the flow characteristics. The technical characteristics include equipment business flow data tracking based on deep packet detection and deep flow detection, feature extraction based on various algorithms, hierarchical extraction of feature information and the like. The overall structure is shown in fig. 1. Specifically, the automatic extraction method of the embodiment of the invention comprises the following aspects:

device traffic data detection

For flow detection of mass terminal equipment, a mode of combining Deep Packet Inspection (DPI) technology and Deep Flow Inspection (DFI) technology is adopted. Mainly two elements are involved: (1) and setting a custom management and control system platform for configuring depth detection rules, monitoring flow, identifying and classifying flow and executing flow control strategies. (2) And establishing a rule base and a strategy definition for identifying and classifying the traffic of the terminal equipment of the Internet of things. The rule base mainly contains signature rules or pattern matching rules of specific protocols and depth detection technologies, and is used for identifying specific traffic types. The policy definition may include a flow control policy, a security policy, and an optimization policy for performing corresponding control and optimization operations based on the identified traffic type.

In the flow management platform, the captured internet of things terminal flow is analyzed through setting rules, and key information such as a source IP address, a target IP address, a port number, a protocol type and the like is extracted. And analyzing the effective load of the data packet from the application layer by monitoring the flow of the terminal equipment of the Internet of things, identifying and classifying different types of flows, and acquiring the communication condition between the equipment and the cloud platform or other equipment.

When creating the rule base, traffic identification needs to be performed on a series of network data packets sent from or to the MAC address by the internet of things device. The network traffic of the mass terminal device may change due to different factors such as environmental conditions or configuration parameters, that is, the traffic patterns of the same internet of things terminal device may be slightly different, which needs to be fed back in the signature mechanism.

For this case, embodiments of the present invention employ a Nilsimsa algorithm that generates a hash output, and similar inputs generate similar hashes. And carrying out local segmentation on the traffic of the terminal equipment, carrying out hash calculation on the segments, and generating a digest with fixed length. And compares the similarity of the two digests to evaluate the similarity of the data. The signature algorithm is set in the rule base of the management platform, and a set of signature overall operations is generated for each device to be analyzed, as shown in fig. 2. For each device, a Nilsima hash is generated from the network traffic of the device. It then stores the hash of the device in a signature database. If the traffic signature comes from a new terminal device, the device may be identified by sorting according to the hash similarity score. That is, when a new device joins the network, the Nilsima hash of the device is calculated from the network traffic and compared to the hash values in the database, the tag set for the new device having the highest average hash similarity score.

As shown in fig. 2, all the device traffic data streams to be analyzed are passed through a nilsima hash generator to generate corresponding device signatures and stored in a corresponding signature database. And (3) through hash comparison and sorting through the similarity of the new fingerprint signatures, returning the highest-ranking device as a prediction result.

(II) feature hierarchical extraction

In order to improve efficiency by utilizing network edge calculation and reduce the dependence of equipment on the cloud, equipment characteristic information is divided into two main types, namely primary characteristics and depth characteristics, node equipment at the edge is used for local primary characteristic extraction, and depth characteristic extraction is completed at the cloud. Namely, a hierarchical extraction framework is provided for carrying out function division, shallow features of local extraction equipment are used for reducing the amount of original data transferred to a cloud end, reducing the network bandwidth pressure and improving the system expansibility, and the shallow features extracted locally can also meet the actual requirements of part of applications, so that analysis control is realized locally and the dependence of the system on the cloud end is reduced. And the time-consuming feature recognition model calculation is executed at the cloud, so that the cloud computing capability is fully applied, and the influence of network communication on time delay is reduced. The hierarchical extraction framework provided by the invention has stronger expansibility, and can realize the increase of the system capacity through the expansion of the edge nodes.

The implementation mode provided by the invention is as follows: setting a feature classification rule, classifying and labeling feature information, calculating the feature information marked as shallow layers by local equipment at the edge, and uploading the feature information to the cloud end, wherein the cloud end does not calculate the shallow layer feature information. And after the feature information which is not marked as the shallow layer is uploaded to the cloud, the terminal feature recognition model in the above steps is called by the cloud to recognize and extract. Aiming at different terminal equipment configurations, the grading rule can be adjusted to realize the custom marking of the equipment characteristic information, and the shallow layer or deep layer grading type of the equipment characteristic information is adaptively adjusted according to the execution states, performance indexes and the like of the equipment and the cloud.

The embodiment of the invention mainly adopts a feature classification rule to classify and mark the features, and determines partial features as shallow features so as to realize the segmentation of the edge end and cloud tasks. And (3) calculating and extracting a returned result only at the edge of the shallow layer feature, so that cloud computing pressure is reduced. The rules of the method can be adjusted according to different equipment configurations, the self-definition of feature classification is realized, the feature classification strategy can be dynamically adjusted according to equipment and cloud states, the self-adaptive optimization is realized, the division of tasks of the side and the cloud is realized, and the advantages are respectively exerted. The rule customization and dynamic adjustment in the mode improves the configurability and the adaptability of the system, and has certain advantages in terms of calculation efficiency and real-time performance. Namely, the method is a dynamic feature classification rule design method, and realizes intelligent collaborative computing of edges and cloud, and specifically comprises the following sub-steps:

step 1: selecting a representative device, and collecting data sets of various types of characteristics within a certain period of time;

step 2: designing a characteristic representation mode, such as vectors, graphs and the like, and extracting characteristic essential attribute information;

step 3: performing unsupervised learning on the feature data set by adopting a deep learning algorithm, and automatically learning a feature space structure;

step 4: based on the learning result, constructing a preliminary static grading rule: for example, the threshold value of the total power change window is 1 minute, and the corresponding relevant characteristic is shallow;

step 5: in practical application, the edge equipment reports new characteristics and evaluation results in real time; for example, the intelligent edge transformer reports voltage and current values once in 1 minute, and the prediction error deviates from the true value;

step 6: the cloud periodically collects samples and retrains the rule model together with the historical data;

step 7: the model outputs adjustment suggestions, such as changing a certain characteristic from deep layer to shallow layer; for example, after learning, determining the voltage, position and other terms as shallow features;

step 8: and updating the edge and cloud rules, and dynamically adjusting the feature classification strategy.

Compared with the static rule, the dynamic feature classification rule can better meet the diversified scenes of the Internet of things, and is not limited to the artificial setting feature classification rule.

And (3) performing feature engineering modeling on the labels defined in the steps, analyzing the abnormal characteristics of each label, performing correlation analysis on error data, and repeatedly modeling. And screening characteristic values with higher correlation degree from large-scale abnormal labels, constructing a plurality of pattern recognition models according to the correlation between the characteristic values and data, and customizing and expanding algorithms according to requirements, wherein the algorithms comprise KNN clustering, BP neural networks, decision trees and the like. And weight division is carried out according to the accuracy, so that the learning is further strengthened. The feature extraction flow is shown in fig. 3.

The feature extraction of the terminal equipment is carried out on the basis of the defined label and mainly comprises the following steps: and (5) classifying the characteristics. In the feature classification, the importance and relevance of each feature to the problem is evaluated. By statistical algorithms of analysis of variance or feature selection algorithms (e.g., chi-square test or information gain). Those features that are most relevant to the problem are selected and redundant or irrelevant features are deleted. In a rule base of the custom management platform, selecting the algorithms according to the needs; and (5) extracting characteristics. The extraction method uses Principal Component Analysis (PCA). And converting the high-dimensional data into low-dimensional data, namely, maintaining main information in the original data while realizing data dimension reduction. In the feature extraction process, related to algorithm application, different algorithms are selected to perform feature clustering or classification according to specific problem types, a KNN clustering algorithm is used for clustering the clustering problems based on the distance between data points, and similar data points are grouped together. Classification problems use BP neural networks or decision tree algorithms. The BP neural network learns complex relationships between data through training and classifies new data. Decision tree algorithms classify by building a tree structure, with each branch representing a feature value.

(III) feature extraction Automation

And (3) automating the flow of the feature recognition model realized by the steps so as to integrate with other systems or applications, realize automatic execution of feature extraction, automatic output and integration of results, and improve the working efficiency and the instantaneity of data processing. The method comprises the following steps:

and (3) model derivation, namely serializing the feature extraction model with the training and optimization completion into a model object, and independently separating the model from a training environment. Model integration, the derived feature extraction model is integrated into independent service, integrated into an application program or a system, and an API interface is provided for other application programs to call. And accessing the data stream, installing an agent or a sensor on the internet of things equipment, and transmitting the data stream to an automation system by using a network protocol. Automated workflow orchestration, using an automated flow management tool, defines workflow and task dependencies for feature extraction applications. And automatically triggering the feature extraction task according to the actual service condition, and managing the execution and state of the task. And outputting and integrating the results, outputting the extracted characteristic results to a specified database, and analyzing and processing the characteristic information at any time. Monitoring and alarming, and setting a monitoring and alarming mechanism. By monitoring the execution state, performance index and log of the feature extraction task, the abnormal situation is found in time, and an alarm notification is triggered to perform timely fault detection and repair.

In the above result output and integration steps, the extracted feature result is output to a specified database so as to analyze and process the feature information comprehensively for a long time;

(IV) adaptive adjustment

The characteristic results extracted in the process can be fed back to corresponding Internet of things equipment in real time, so that the equipment can carry out self-adaptive modulation according to the characteristic results. The method comprises the following specific steps:

the feature extraction system trains a model according to the preprocessed data and extracts various operation features of the equipment in real time, such as performance indexes of CPU utilization rate, memory use condition and the like.

And sending the extracted characteristic results to the equipment management system in real time through a communication network or a message queue and the like. When extracting the result, a type mark needs to be added to each feature record, and the type mark can be preset into several categories, for example, performance categories: cpu utilization, memory utilization, etc., error class: exception code, error log, etc., using pattern classes: access frequency, traffic statistics, etc., and then send the feature results to the device management system through the message transfer server. For example, the feature type preset flag is as follows:

performance class characteristics:

{ "name": "cpu utilization", "type": "performance" }

{ "name": "memory usage", "type": "performance" }

{ "name": "network bandwidth", "type": "performance" }

Error class feature:

{ "name": "exception error code", "type": "error" }

{ "name": "error log", "type": "error" }

{ "name": "failure times", "type": "error" }

Using pattern class features:

{ "name": "daily access traffic", "type": "usage" }

{ "name": "number of concurrent users", "type": "usage" }

{ "name": "on-line time distribution", "type": "usage" }

Deployment class feature:

{ "name": "CPU core number", "type": "depth" } of

{ "name": "gantry position", "type": "depth" } of

{ "name": "software and hardware version", "type": "depth" } of

Environmental class characteristics:

{ "name": "average temperature", "type": "environment" }

{ "name": "Fan speed", "type": "environment" }

{ "name": "Power type", "type": "environment" }

After receiving the feature result, the device management system analyzes the message body in a reverse sequencing way, obtains the feature result and the type mark thereof, routes the feature result, and distributes the feature result to the subsystem interface of the corresponding device according to different feature types: the performance characteristics such as CPU and memory are fed back to the equipment resource scheduling system; the error class characteristics such as error reporting codes are fed back to the bug repairing system; pattern features such as flow peaks are used to feed back to the device pricing system, etc.

Each subsystem automatically decides the adjustment behavior according to the characteristic result: the resource scheduling system optimizes CPU core allocation or memory utilization strategies and the like: the error repair system periodically issues code patch updates to solve the problem; the pricing system adjusts price policies, etc., based on the traffic characteristics.

The device adopts an automatic configuration mode, and the self-optimization is carried out by applying the adjustment result in real time. The log system re-collects the later data, observes the adjustment effect, and loops the feature extraction and optimization process. In this step, the following steps are taken:

the log collection system re-collects the device log according to a certain time sequence (e.g., every hour). 2. And comparing with the original log, and extracting the newly added log and the modified log. 3. The log processing system performs the same preprocessing workflow on the newly added log. 4. The extraction system updates the original model in real time according to the newly added data, and extracts the newly added characteristic index. 5. And comparing the historical characteristic with the historical characteristic, and evaluating whether the adjusted new characteristic value tends to be an expected effect. 6. And obtaining the optimization degree of the key index through characteristic effect monitoring, log analysis and the like. 7. And feeding back the monitoring result to the equipment management system. 8. And the equipment management system evaluates success and deficiency of the last round of adjustment strategy according to the monitoring result. 9. And generating a new optimization scheme by machine learning the lifting model. 10. The management system issues a new round of adjustment strategy to the device end for implementation.

The invention adopts deep packet detection and deep flow detection based on deep analysis of protocol messages, adopts a corresponding signature mechanism according to the characteristics of hash input/output similarity, and marks equipment terminals. And then, carrying out characteristic information grading and engineering modeling on the defined labels, analyzing the characteristics of each label, carrying out correlation analysis on the characteristics and data of each label, constructing a mode identification model consisting of a plurality of algorithms, and carrying out automatic workflow arrangement on the whole process, thereby realizing automatic extraction of characteristics of the internet of things terminal.

When the scheme is implemented, circulation caused by abnormality can be possibly encountered, an abnormality detection and rollback mechanism needs to be designed, and a target index threshold value and a time limit are set. For example, the CPU utilization rate target is 45%, if the CPU utilization rate is not reduced within 2 hours, the strategy is considered to be abnormal, when the abnormality is detected, the strategy is restarted by the last strategy parameter or initial parameter, the monitoring effect is continued after the strategy is restarted, if the problem is eliminated, the optimization is continued, and otherwise, the strategy is restarted by one stage. If abnormal data are encountered, the subsequent learning is not included, and the model is prevented from learning to influence the subsequent optimization due to abnormal conclusion. Setting a rollback threshold, if n times of rollbacks cannot be solved, manual intervention is needed, and an abnormal log and a problem root cause are recorded.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The solutions in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java, and an transliterated scripting language JavaScript, etc.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

Claims (10)

1. The automatic extraction method of the electric power internet of things terminal characteristics is characterized by comprising the following steps of:

s1, identifying flow information of an electric power internet of things terminal by adopting a depth detection technology, and storing an equipment signature obtained by identification into a corresponding signature database through hash mapping;

s2, setting a feature classification rule, classifying and labeling the equipment feature information, and dividing the equipment feature information into shallow features and depth features; aiming at the flow information of the electric power internet of things terminal, shallow layer characteristics of equipment are extracted at edge nodes, and depth characteristics of the equipment are extracted at a cloud;

s3, analyzing the abnormal characteristics of each label, performing correlation analysis on the characteristics extracted in the step S2, and screening out characteristic values with correlation higher than a correlation threshold; performing dimension reduction processing on the screened characteristic values, clustering or classifying the characteristic values after the dimension reduction processing, and constructing a characteristic extraction model with multiple modes;

and S4, combining the workflow automation programming step S3 to construct a multi-mode feature extraction model to finish the automatic extraction of the flow features.

2. The method for automatically extracting the characteristics of the electric power internet of things terminal according to claim 1, wherein in step S1, the process of identifying the flow information of the electric power internet of things terminal by adopting a depth detection technology and storing the identified device signature into a corresponding signature database through hash mapping comprises the following steps:

analyzing the flow information of the captured electric power internet of things terminal, and extracting key information including a source IP address, a target IP address, a port number and a protocol type;

carrying out local segmentation on flow information of the electric power Internet of things terminal obtained through analysis, carrying out hash calculation on all fragments, generating a hash value abstract with fixed length, and storing the generated hash value abstract in a signature database; the similarity of the flow information of the electric power internet of things terminal is evaluated by comparing the similarity of the two abstracts;

when a new electric power internet of things terminal joins the network, according to Nilsima hash of the network flow computing device, and comparing the Nilsima hash with the hash value abstract stored in the signature database, setting a device type label with the highest average hash similarity score for the new electric power internet of things terminal.

3. The automated extraction method of electric power internet of things terminal features of claim 2, wherein for each electric power internet of things terminal, a Nilsima hash is generated for the network communication flow of the electric power internet of things terminal, and the generated Nilsima hash is stored in a signature database.

4. The automatic extraction method of the electric power internet of things terminal characteristics according to claim 1, wherein in the step S2, characteristic classification rules are set, equipment characteristic information is classified and labeled, and the equipment characteristic information is divided into shallow characteristics and depth characteristics; aiming at the flow information of the electric power internet of things terminal, shallow layer characteristics of equipment are extracted at edge nodes, and depth characteristics of the equipment are extracted at a cloud end, wherein the method comprises the following steps:

s21, selecting a representative electric power internet of things terminal device, collecting characteristic data of all selected devices within a certain time period, and constructing to obtain a characteristic data set;

s22, designing a feature representation mode according to the feature essential attribute information;

s23, performing unsupervised learning on the feature data set by adopting a deep learning algorithm, and automatically learning a feature space structure;

s24, constructing a preliminary static grading rule based on the learning result of the step S23 to obtain a rule model;

s25, reporting the extracted new features and the evaluation results in real time by the edge node to serve as a group of samples;

s26, periodically collecting samples by the cloud, retraining the rule model together with the historical sample data, dynamically adjusting the rule model according to the training result, and modifying the classification result of part of the characteristics.

5. The automated extraction method of characteristics of an electric power internet of things terminal according to claim 4, wherein in step S2, in step S26, a rule model is dynamically adjusted in combination with an equipment operation state, a cloud operation state and a training result.

6. The automatic extraction method of the characteristics of the electric power internet of things terminal according to claim 1, wherein in the step S3, the abnormal characteristics of each label are analyzed, the correlation analysis is carried out on the characteristics extracted in the step S2 through a statistical algorithm or a characteristic selection algorithm of variance analysis, and the characteristic value with the correlation higher than a correlation threshold value is screened out; and then adopting principal component analysis to perform dimension reduction treatment on the screened characteristic values.

7. The automated extraction method of electric power internet of things terminal features according to claim 1, wherein in step S3, clustering is performed based on distances between data points using KNN clustering algorithm, grouping similar data points together; the new data is classified using a BP neural network or decision tree algorithm.

8. The automated extraction method of characteristics of an electric power internet of things terminal according to claim 1, wherein in step S4, the process of completing the automated extraction of the flow characteristics by combining the workflow automation orchestration step S3 to construct the obtained multi-mode characteristic extraction model comprises the following steps:

s41, serializing the feature extraction model after training and optimization into a model object, and independently extracting the feature extraction model from a training environment;

s42, integrating the derived feature extraction model into independent service, integrating the independent service into an automation system, and providing an API interface for other application programs to call;

s43, installing an agent or a sensor on the electric power internet of things terminal equipment, and transmitting the data stream to an automation system by using a network protocol;

s44, defining a workflow and task dependency relationship of a feature extraction application by using an automatic flow management tool, automatically triggering a feature extraction task according to actual business conditions, extracting features by adopting a feature extraction model, and managing the execution state of the extraction task in real time;

s45, outputting the extracted characteristic result to a specified database for storage;

s46, judging whether an abnormal situation occurs or not by monitoring the execution state, the performance index and the log of the feature extraction task, and triggering an alarm notification if the abnormal situation occurs.

9. The automated extraction method of electrical thing networking terminal features of claim 8, further comprising: analyzing the characteristic results in the appointed database, and feeding back the analysis results to the corresponding electric power internet of things terminal in real time, so that the electric power internet of things terminal carries out self-adaptive modulation according to the characteristic results; the method specifically comprises the following substeps:

extracting various operation characteristics of the electric power Internet of things terminal in real time;

the extracted characteristic results are sent to the equipment management system in real time through a communication network or a message queue and the like; wherein, when extracting the result, a type mark is added to each feature record, and the type mark comprises: performance class features, error class features, usage pattern class features, deployment class features, and environment class features;

after receiving the characteristic results, the equipment management system analyzes the message body through reverse serialization, obtains the characteristic results and the type marks thereof, routes the characteristic results, distributes the characteristic results to subsystem interfaces of corresponding equipment according to different characteristic types, and each subsystem automatically executes adjustment actions according to the characteristic results, so that the electric power Internet of things terminal adopts an automatic configuration mode and automatically optimizes by applying the adjustment results in real time;

the log system re-collects the later data, observes the adjustment effect, and loops the feature extraction and optimization process.

10. The automated extraction method of electrical internet of things terminal features of claim 9, wherein the log system re-collects later data, observes the effects of adjustments, and loops the process of feature extraction and optimization comprising the steps of:

the log collecting system re-collects the equipment operation log according to a certain time sequence, compares the collected equipment operation log with the original log, and extracts a newly added log and a modified log;

the extraction system updates the original model in real time according to the newly added data, and extracts newly added characteristic indexes; comparing the newly added characteristic index with the historical characteristic, and evaluating whether the adjusted newly added characteristic index tends to an expected effect; the optimization degree of key indexes is obtained through characteristic effect monitoring and log analysis, and monitoring results are fed back to the equipment management system;

the equipment management system evaluates whether the effect of the last round of adjustment strategy reaches the expected purpose according to the monitoring result; if the adjustment strategy is not achieved, a new optimization scheme is generated through the machine learning lifting model, and a new round of adjustment strategy is issued to the electric power Internet of things terminal for implementation by the equipment management system.