CN117041121A

CN117041121A - Internet of things anomaly monitoring method and system based on data mining

Info

Publication number: CN117041121A
Application number: CN202311280569.3A
Authority: CN
Inventors: 胡涛; 刘跃华; 陈廉之
Original assignee: Chengdu Silent Communication Technology Co ltd
Current assignee: Sichuan Everything Technology Co ltd
Priority date: 2023-10-07
Filing date: 2023-10-07
Publication date: 2023-11-10
Anticipated expiration: 2043-10-07
Also published as: CN117041121B

Abstract

The invention provides an anomaly monitoring method and system of the Internet of things based on data mining, and relates to the technical field of artificial intelligence. In the invention, a system operation monitoring data cluster corresponding to the system of the Internet of things to be monitored is determined; determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster; and carrying out operation anomaly analysis operation on the system operation monitoring data cluster by utilizing the optimized anomaly analysis network so as to analyze a system operation identification data cluster corresponding to the system of the Internet of things to be monitored. Based on the above, the reliability of monitoring the abnormality of the internet of things can be improved to a certain extent.

Description

Internet of things anomaly monitoring method and system based on data mining

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to an Internet of things anomaly monitoring method and system based on data mining.

Background

By means of the information sensing equipment, any article is connected with the Internet according to the agreed protocol for information exchange and communication, so as to realize intelligent identification, positioning, tracking, monitoring and management. In popular terms, the internet of things is the internet of things, and comprises two layers of meanings: firstly, the Internet of things is an extension and expansion of the Internet, and the core and the foundation of the Internet are still the Internet; secondly, the user side of the Internet of things not only comprises people, but also comprises articles, and the Internet of things realizes the exchange and communication of information among people, articles and articles. Because the application range of the internet of things is relatively wide, abnormal monitoring needs to be carried out on the internet of things, but in the prior art, the problem of low reliability exists.

Disclosure of Invention

Accordingly, the present invention aims to provide a method and a system for monitoring the abnormality of the internet of things based on data mining, so as to improve the reliability of monitoring the abnormality of the internet of things to a certain extent.

In order to achieve the above purpose, the embodiment of the present invention adopts the following technical scheme:

an internet of things anomaly monitoring method based on data mining, comprising the following steps:

determining a system operation monitoring data cluster corresponding to the system of the Internet of things to be monitored, wherein the system operation monitoring data in the system operation monitoring data cluster comprises a system operation log;

determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster;

and carrying out operation anomaly analysis operation of the Internet of things on the system operation monitoring data cluster by utilizing the optimized anomaly analysis network so as to analyze a system operation identification data cluster corresponding to the system of the Internet of things to be monitored, wherein the system operation identification data cluster comprises a plurality of system operation identification data which are used for reflecting operation anomaly prediction information of the system of the Internet of things to be monitored on a plurality of operation monitoring layers.

In some preferred embodiments, in the data mining-based method for monitoring abnormal internet of things, the system operation monitoring data cluster includes system operation monitoring data corresponding to at least one data dimension other than the preset data dimension; the optimized anomaly analysis network is formed by performing network updating operation on candidate anomaly analysis networks based on later-stage exemplary data, the later-stage exemplary data comprises later-stage exemplary system operation monitoring data clusters and exemplary identification data corresponding to preset data dimensions, the later-stage exemplary system operation monitoring data clusters comprise system operation monitoring data corresponding to the preset data dimensions, the candidate anomaly analysis network is formed by performing network updating operation on original anomaly analysis networks based on earlier-stage exemplary data, and the earlier-stage exemplary data comprises earlier-stage exemplary system operation monitoring data clusters corresponding to a plurality of exemplary data dimensions and corresponding exemplary identification data clusters;

The step of performing operation anomaly analysis operation on the system operation monitoring data cluster by using the optimized anomaly analysis network to analyze the system operation identification data cluster corresponding to the system of the internet of things to be monitored comprises the following steps:

loading the system operation monitoring data cluster to a feature space mapping unit included in the optimized anomaly analysis network, and performing feature space mapping operation on the system operation monitoring data cluster by using the feature space mapping unit to form each operation space mapping feature representation corresponding to each data dimension;

performing mapping feature integration operation on each operation space mapping feature representation corresponding to each data dimension by using a mapping feature integration unit included in the optimized anomaly analysis network so as to output operation feature representation to be analyzed corresponding to the Internet of things system to be monitored;

and matching the system operation identification data corresponding to the to-be-analyzed operation characteristic representation in the to-be-identified data cluster corresponding to the preset data dimension by utilizing the optimized anomaly analysis network to form a system operation identification data cluster corresponding to the to-be-monitored Internet of things system, wherein the preset data dimension belongs to the dimension of the to-be-monitored Internet of things system in data exchange, and the system operation identification data cluster at least comprises system operation identification data corresponding to data exchange loss and system operation identification data corresponding to data exchange distortion.

In some preferred embodiments, in the method for monitoring abnormal internet of things based on data mining, the forming process of the post-stage exemplary data includes:

determining a preceding Internet of things operation information cluster corresponding to a preceding time interval, and determining a following Internet of things operation information cluster which is arranged in a following time interval based on the preset data dimension, wherein the following time interval belongs to a time interval after the preceding time interval, and the preceding Internet of things operation information cluster comprises Internet of things system operation information corresponding to the preset data dimension;

analyzing and forming a later-period exemplary system operation monitoring data cluster included in later-period exemplary data based on the previous Internet of things operation information cluster;

and analyzing and forming the exemplary identification data corresponding to the preset data dimension included in the later exemplary data based on the running information cluster of the internet of things.

In some preferred embodiments, in the method for monitoring anomalies of internet of things based on data mining, the network updating process of the candidate anomaly analysis network includes:

determining configured master screening parameters and slave screening parameters;

and based on the later-stage exemplary data, carrying out network updating operation on the candidate abnormal analysis network, screening system operation monitoring data corresponding to preset data dimensions in the later-stage exemplary system operation monitoring data cluster based on the main screening parameter, and screening system operation monitoring data corresponding to other data dimensions in the later-stage exemplary system operation monitoring data cluster based on the auxiliary screening parameter.

In some preferred embodiments, in the method for monitoring abnormal internet of things based on data mining, the forming process of the preliminary exemplary data includes:

determining an example Internet of things operation information cluster corresponding to each example data dimension respectively;

determining identification data corresponding to each piece of the operation information of the Internet of things in each piece of the operation information clusters of the Internet of things to form an identification data exemplary cluster corresponding to each piece of the data dimension;

analyzing importance characterization parameters of the identification data in each identification data exemplary cluster based on operation duration and operation information validity parameters corresponding to the operation information of the exemplary Internet of things corresponding to the identification data in each identification data exemplary cluster;

analyzing an identification data screening cluster corresponding to the early-stage exemplary data based on the importance characterization parameters of each identification data in each identification data exemplary cluster;

determining screening identification data of a target number in the identification data screening cluster to determine target exemplary identification data corresponding to the early-stage exemplary data, and forming an exemplary identification data cluster corresponding to the early-stage exemplary data based on the target exemplary identification data;

And determining a front-stage exemplary system operation monitoring data cluster corresponding to the front-stage exemplary data based on other screening identification data which is not determined to be target exemplary identification data in the identification data screening cluster and the exemplary Internet of things operation information cluster.

In some preferred embodiments, in the method for monitoring abnormal internet of things based on data mining, the step of determining a target number of screening identification data in the identification data screening cluster to determine target exemplary identification data corresponding to the previous exemplary data, and forming an exemplary identification data cluster corresponding to the previous exemplary data based on the target exemplary identification data includes:

in the identification data screening cluster, determining screening identification data of target quantity so as to determine target exemplary identification data corresponding to the early exemplary data;

marking the target exemplary identification data to be marked as related exemplary identification data corresponding to the previous exemplary data;

determining non-relevant exemplary identification data, wherein the non-relevant exemplary identification data is based on at least one of first exemplary identification data and second exemplary identification data, and the first exemplary identification data belongs to relevant exemplary identification data corresponding to other exemplary data except the previous exemplary data; the second exemplary identification data belongs to identification data in a pre-built identification database;

And determining an exemplary identification data cluster corresponding to the early exemplary data based on the related exemplary identification data and the non-related exemplary identification data.

In some preferred embodiments, in the data mining-based internet of things anomaly monitoring method, the mapping feature integration unit includes a feature internal integration unit, a feature external integration unit, a feature association integration unit, and a feature linear integration unit;

the step of performing mapping feature integration operation on each operation space mapping feature representation corresponding to each data dimension by using a mapping feature integration unit included in the optimized anomaly analysis network to output an operation feature representation to be analyzed corresponding to the system of the internet of things to be monitored includes:

utilizing the feature internal integration unit to integrate each operation space mapping feature representation corresponding to each data dimension respectively so as to output an internal mapping feature representation corresponding to each data dimension;

utilizing the characteristic external integration unit to integrate each internal mapping characteristic representation so as to output an external mapping characteristic representation corresponding to the Internet of things system to be monitored;

Performing, by the feature association integration unit, an association analysis operation on local feature representations in the external map feature representations to output corresponding association analysis feature representations, the external map feature representations including a plurality of local feature representations, an object of the association analysis operation including at least two local feature representations in the plurality of local feature representations, each of the two local feature representations in the plurality of local feature representations having a same feature representation size, a feature representation parameter in each of the local feature representations belonging to a feature representation parameter in the external map feature representation;

and performing linear integration operation on the external mapping feature representation and the association analysis feature representation by using the feature linear integration unit so as to output an operation feature representation to be analyzed corresponding to the Internet of things system to be monitored.

In some preferred embodiments, in the method for monitoring abnormal internet of things based on data mining, the step of using the feature internal integration unit to integrate each running space mapping feature representation corresponding to each data dimension to output an internal mapping feature representation corresponding to each data dimension includes:

Determining saliency evaluation parameters respectively possessed by each operation space mapping feature representation corresponding to each data dimension by using the feature internal integration unit;

performing weighted integration operation on each running space mapping feature representation corresponding to each data dimension based on each corresponding significance evaluation parameter by using the feature internal integration unit so as to form an internal mapping feature representation corresponding to each data dimension;

and the step of integrating the internal mapping feature representations by using the feature external integration unit to output the external mapping feature representation corresponding to the to-be-monitored internet of things system includes:

determining saliency evaluation parameters respectively corresponding to the internal mapping feature representations by utilizing the feature external integration unit;

and carrying out weighted integration operation on each internal mapping feature representation based on the corresponding significance evaluation parameters by utilizing the feature external integration unit so as to form an external mapping feature representation corresponding to the Internet of things system to be monitored.

In some preferred embodiments, in the data mining-based internet of things anomaly monitoring method, the step of determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster includes:

Determining a candidate abnormality analysis network, wherein the candidate abnormality analysis network is formed by performing network updating operation on an original abnormality analysis network based on early-stage exemplary data, and the early-stage exemplary data comprises early-stage exemplary system operation monitoring data clusters corresponding to a plurality of exemplary data dimensions and corresponding exemplary identification data clusters;

determining later-stage exemplary data, wherein the later-stage exemplary data comprises later-stage exemplary system operation monitoring data clusters and exemplary identification data corresponding to preset data dimensions, and the earlier-stage exemplary system operation monitoring data clusters comprise system operation monitoring data corresponding to the preset data dimensions;

loading the later-stage exemplary system operation monitoring data cluster to a feature space mapping unit of the candidate anomaly analysis network, so as to perform feature space mapping operation on the later-stage exemplary system operation monitoring data cluster by utilizing the feature space mapping unit to form each operation space mapping feature representation corresponding to each data dimension;

performing mapping feature integration operation on each operation space mapping feature representation corresponding to each data dimension by using a mapping feature integration unit included in the candidate anomaly analysis network so as to output a corresponding operation feature representation to be analyzed;

And based on the to-be-analyzed running characteristic representation and the exemplary identification data corresponding to the preset data dimension, performing network updating operation on the candidate abnormal analysis network to form an optimized abnormal analysis network corresponding to the preset data dimension.

The embodiment of the invention also provides a data mining-based Internet of things abnormality monitoring system, which comprises a processor and a memory, wherein the memory is used for storing a computer program, and the processor is used for executing the computer program so as to realize the data mining-based Internet of things abnormality monitoring method.

The method and the system for monitoring the abnormality of the Internet of things based on the data mining can determine the system operation monitoring data cluster corresponding to the system of the Internet of things to be monitored; determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster; and carrying out operation anomaly analysis operation on the system operation monitoring data cluster by utilizing the optimized anomaly analysis network so as to analyze a system operation identification data cluster corresponding to the system of the Internet of things to be monitored. Based on the foregoing, because the analyzed system operation identification data cluster includes a plurality of system operation identification data, the content for representing the abnormal operation is richer, so that the reliability of monitoring the abnormal operation of the internet of things can be improved to a certain extent, and the problem of low reliability of monitoring the abnormal operation of the internet of things in the prior art is solved.

In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

Fig. 1 is a structural block diagram of an anomaly monitoring system of internet of things based on data mining according to an embodiment of the present invention.

Fig. 2 is a schematic flow chart of steps included in the data mining-based internet of things anomaly monitoring method according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of each module included in the data mining-based internet of things anomaly monitoring device according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

As shown in fig. 1, the embodiment of the invention provides an internet of things anomaly monitoring system based on data mining. The internet of things anomaly monitoring system may include a memory and a processor.

In particular, the memory and the processor are electrically connected directly or indirectly to enable transmission or interaction of data. For example, electrical connection may be made to each other via one or more communication buses or signal lines. The memory may store at least one software functional module (computer program) that may exist in the form of software or firmware. The processor may be configured to execute the executable computer program stored in the memory, so as to implement the data mining-based internet of things anomaly monitoring method provided by the embodiment of the present invention (as described below).

Specifically, in some possible embodiments, the Memory may be, but is not limited to, random access Memory (Random Access Memory, RAM), read Only Memory (ROM), programmable Read Only Memory (Programmable Read-Only Memory, PROM), erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), and the like. The processor may be a general purpose processor including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a System on Chip (SoC), etc.; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In particular, in some possible embodiments, the data mining-based internet of things anomaly monitoring system may be a server with data processing capabilities.

With reference to fig. 2, the embodiment of the invention further provides a data mining-based internet of things anomaly monitoring method, which can be applied to the data mining-based internet of things anomaly monitoring system. The method steps defined by the flow related to the data mining-based internet of things anomaly monitoring method can be realized by the data mining-based internet of things anomaly monitoring system.

The specific flow shown in fig. 2 will be described in detail.

Step S110, determining a system operation monitoring data cluster corresponding to the system of the Internet of things to be monitored.

In the embodiment of the invention, the data mining-based abnormal monitoring system of the internet of things can determine the system operation monitoring data cluster corresponding to the system of the internet of things to be monitored. The system operation monitoring data in the system operation monitoring data cluster includes a system operation log, i.e. an operation process for recording an operation process of the system of the internet of things to be monitored, such as an operation process of the internet of things device, such as each internet of things terminal, internet of things gateway, etc., for example, the XXX operation may be performed by the XXX device in XXX time.

And step S120, determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster.

In the embodiment of the invention, the data mining-based abnormal monitoring system of the internet of things can determine an optimized abnormal analysis network corresponding to the system operation monitoring data cluster. That is, the optimized anomaly analysis network is formed by network learning based on other operation monitoring data associated with the system operation monitoring data cluster.

And step S130, performing operation of analyzing the abnormal operation of the Internet of things on the system operation monitoring data cluster by utilizing the optimized abnormal analysis network so as to analyze the system operation identification data cluster corresponding to the system of the Internet of things to be monitored.

In the embodiment of the invention, the data mining-based internet of things anomaly monitoring system can perform the operation anomaly analysis operation of the internet of things on the system operation monitoring data cluster by utilizing the optimized anomaly analysis network so as to analyze the system operation identification data cluster corresponding to the system of the internet of things to be monitored. The system operation identification data cluster comprises a plurality of system operation identification data, wherein the plurality of system operation identification data are used for reflecting operation abnormality prediction information of the to-be-monitored internet of things system on a plurality of operation monitoring layers, such as whether the to-be-monitored internet of things system has abnormality or not.

Based on the foregoing, that is, the foregoing steps S110, S120, and S130, since the analyzed system operation identification data cluster includes a plurality of system operation identification data, the content for characterizing the abnormal operation is more abundant, so that the reliability of monitoring the abnormal operation of the internet of things can be improved to a certain extent, thereby improving the problem of low reliability of monitoring the abnormal operation of the internet of things in the prior art.

Specifically, in some possible embodiments, the step S120 above, that is, the step of determining the optimized anomaly analysis network corresponding to the system operation monitoring data cluster, may further include the following detailed implementation details:

determining a candidate abnormality analysis network, wherein the candidate abnormality analysis network is formed by carrying out network updating operation on an original abnormality analysis network based on early-stage exemplary data, the early-stage exemplary data comprises early-stage exemplary system operation monitoring data clusters corresponding to a plurality of exemplary data dimensions and corresponding exemplary identification data clusters, namely, a data mapping relation between the system operation monitoring data clusters and the exemplary identification data clusters is learned;

Loading the later-stage exemplary system operation monitoring data cluster to a feature space mapping unit of the candidate anomaly analysis network, so as to perform feature space mapping operation on the later-stage exemplary system operation monitoring data cluster by using the feature space mapping unit to form each operation space mapping feature representation corresponding to each data dimension, as described in the previous related description;

performing mapping feature integration operation on each operation space mapping feature representation corresponding to each data dimension by using a mapping feature integration unit included in the candidate anomaly analysis network so as to output a corresponding operation feature representation to be analyzed, as described in the previous related description;

and carrying out network updating operation on the candidate abnormal analysis network based on the operation characteristic representation to be analyzed and the exemplary identification data corresponding to the preset data dimension so as to form an optimized abnormal analysis network corresponding to the preset data dimension, namely updating and optimizing network parameters included in the candidate abnormal analysis network based on the difference between the operation characteristic representation to be analyzed and the exemplary identification data corresponding to the preset data dimension.

Specifically, in some possible embodiments, the step of performing a network update operation on the candidate anomaly analysis network based on the to-be-analyzed running feature represents the exemplary identification data corresponding to the preset data dimension to form an optimized anomaly analysis network corresponding to the preset data dimension may further include the following detailed implementation contents:

determining an actual characterization parameter of the exemplary identification data corresponding to the preset data dimension based on the relationship between the exemplary identification data corresponding to the preset data dimension and the related exemplary identification data and the non-related exemplary identification data, for example, when the exemplary identification data corresponding to the preset data dimension belongs to the related exemplary identification data, the actual characterization parameter may be equal to 1, for example, the actual characterization parameter may be equal to 0, for example, the actual characterization parameter may not have data interaction distortion, when the exemplary identification data corresponding to the preset data dimension belongs to the non-related exemplary identification data;

performing feature space mapping operation, such as encoding, on the exemplary identification data corresponding to the preset data dimension to form a corresponding exemplary identification feature representation;

Calculating a number product of the exemplary identification feature representation and the operation feature representation to be analyzed to output a corresponding feature representation number product, performing an exponential operation on the feature representation number product, and finally, adjusting a result of the exponential operation to form an estimated characterization parameter corresponding to the operation feature representation to be analyzed, for example, a summation calculation may be performed on a result of the exponential operation and a target value, and then a ratio calculation may be performed on a result of the exponential operation and a result of the summation calculation to obtain a corresponding estimated characterization parameter, where the target value may be equal to 1;

determining a first negative correlation value of the estimated characterization parameter and determining a second negative correlation value of the actual characterization parameter, e.g. the sum between the first negative correlation value and the estimated characterization parameter may be equal to the target value, and the sum between the second negative correlation value and the actual characterization parameter may be equal to the target value;

weighting the logarithm result of the estimated characterization parameter based on the actual characterization parameter to form a corresponding first weighting parameter, weighting the logarithm result of the first negative correlation value based on the second negative correlation value to form a corresponding second weighting parameter, and fusing the first weighting parameter and the second weighting parameter to obtain a network update error parameter of the candidate anomaly analysis network; illustratively, a sum value between the first weighting coefficient and the second weighting coefficient may be calculated first, and the network update error parameter may have a negative correlation with the sum value, e.g., the sum value therebetween is equal to 1; in other embodiments, if the exemplary identification data corresponding to the preset data dimension may be a plurality of pieces, such as system operation identification data corresponding to data exchange loss and system operation identification data corresponding to data exchange distortion, average calculation may be further performed on the obtained plurality of network update error parameters to obtain a final network update error parameter, so as to perform subsequent network update operations;

And carrying out network updating operation on the candidate abnormal analysis network along the direction of reducing the network updating error parameter to form an optimized abnormal analysis network corresponding to the preset data dimension.

Specifically, in some possible embodiments, the system operation monitoring data cluster includes system operation monitoring data corresponding to at least one data dimension other than the preset data dimension, that is, the system operation monitoring data cluster includes system operation monitoring data corresponding to at least one other data dimension other than the preset data dimension on the basis of including system operation monitoring data corresponding to the preset data dimension. In addition, the optimized anomaly analysis network is formed by performing a network update operation on a candidate anomaly analysis network based on later-stage exemplary data, wherein the later-stage exemplary data comprises a later-stage exemplary system operation monitoring data cluster and exemplary identification data (used for reflecting corresponding operation anomaly information) corresponding to the preset data dimension, the later-stage exemplary system operation monitoring data cluster comprises system operation monitoring data corresponding to the preset data dimension, and the candidate anomaly analysis network is formed by performing a network update operation on an original anomaly analysis network based on earlier-stage exemplary data, wherein the earlier-stage exemplary data comprises earlier-stage exemplary system operation monitoring data clusters corresponding to a plurality of exemplary data dimensions and corresponding exemplary identification data clusters.

Based on this, step S130 above, that is, the step of performing an operation anomaly analysis operation on the system operation monitoring data cluster by using the optimized anomaly analysis network to analyze the system operation identification data cluster corresponding to the system of the internet of things to be monitored, may further include the following detailed implementation contents:

loading the system operation monitoring data cluster to a feature space mapping unit included in the optimized anomaly analysis network, and performing feature space mapping operation on the system operation monitoring data cluster by using the feature space mapping unit to form each operation space mapping feature representation corresponding to each data dimension, wherein the feature space mapping unit can be a coding network, so that the system operation monitoring data corresponding to each data dimension can be respectively coded to realize feature space mapping, and each operation space mapping feature representation corresponding to each data dimension is obtained;

And matching the system operation identification data corresponding to the to-be-analyzed operation characteristic representation in the to-be-analyzed identification data cluster corresponding to the preset data dimension by utilizing the optimized anomaly analysis network to form a system operation identification data cluster corresponding to the to-be-monitored internet of things system, wherein the preset data dimension belongs to the dimension of data exchange of the to-be-monitored internet of things system (based on the dimension, the analysis can be performed on gateway equipment in the to-be-monitored internet of things system, and thus, the at least one other data dimension can be the dimension of executing information of an instruction issued by a server at the back end to the gateway equipment, the dimension of recording information for carrying out data sharing between the gateway equipment, the dimension of recording information for carrying out cooperative calculation between the gateway equipment, and the like), and the system operation identification data cluster at least comprises system operation identification data corresponding to data loss of data exchange, system operation identification data corresponding to data interaction distortion, and other data selected based on actual requirements.

Specifically, in some possible embodiments, the process of forming the post-exemplary data may further include the following detailed implementation:

Determining a preceding internet of things operation information cluster corresponding to a preceding time interval, and determining a following internet of things operation information cluster which is arranged in a following time interval based on the preset data dimension, wherein the following time interval belongs to a time interval after the preceding time interval (in addition, the specific time interval length of the following time interval and the specific time interval of the preceding time interval is not limited, and the interval between the following time interval and the preceding time interval is not limited), and the preceding internet of things operation information cluster comprises internet of things system operation information corresponding to the preset data dimension, namely, data recorded in a data exchange process performed on a corresponding internet of things system can be expressed by text;

analyzing a later-period exemplary system operation monitoring data cluster included in the formed later-period exemplary data based on the previous internet of things operation information cluster, for example, the previous internet of things operation information cluster can be directly used as the later-period exemplary system operation monitoring data cluster;

based on the operation information cluster of the internet of things, analyzing and forming the exemplary identification data corresponding to the preset data dimension included in the later exemplary data, for example, the operation information cluster of the internet of things can be analyzed, if information carried in the operation information cluster of the internet of things is extracted, such as state information which is reported at the front end and can represent abnormal operation, or state information which is issued at the rear end and can represent abnormal operation, if the corresponding state information is not extracted, manual labeling can be performed to obtain the exemplary identification data.

Specifically, in some possible embodiments, the network update procedure of the candidate anomaly analysis network may further include the following detailed implementation contents:

based on the later-stage exemplary data, carrying out network updating operation on the candidate abnormal analysis network, screening system operation monitoring data corresponding to preset data dimensions in a later-stage exemplary system operation monitoring data cluster based on the main screening parameter, and screening system operation monitoring data corresponding to other data dimensions in the later-stage exemplary system operation monitoring data cluster based on the auxiliary screening parameter; that is, in the process of updating the network of the candidate anomaly analysis network based on the later-stage exemplary data, the system operation monitoring data corresponding to the preset data dimension in the later-stage exemplary system operation monitoring data cluster is selected based on the main screening parameter, namely, the system operation monitoring data corresponding to the preset data dimension in the later-stage exemplary system operation monitoring data cluster is screened out; after discarding the system operation monitoring data corresponding to the preset data dimension, only the system operation monitoring data corresponding to other data dimensions are remained in the later-stage exemplary system operation monitoring data cluster, so that the system operation monitoring data cluster can learn how to map the data of other data dimensions to the system operation identification data cluster of the preset data dimension when the data of the preset data dimension is discarded, and the dependence on the data of the preset data dimension is reduced; on the other hand, in the process of updating the network of the candidate anomaly analysis network based on the later-stage exemplary data, the system operation monitoring data corresponding to other data dimensions in the later-stage exemplary system operation monitoring data cluster is selected based on the slave screening parameter, namely, the system operation monitoring data of the part corresponding to other data dimensions in the later-stage exemplary system operation monitoring data cluster is screened out, and when the system operation monitoring data corresponding to other data dimensions is screened out, the data of one or more data dimensions in the other data dimensions can be selected for screening out.

The specific values of the master screening parameter and the slave screening parameter are not limited, and the master screening parameter is smaller than the slave screening parameter, for example, in a specific embodiment, the system operation monitoring data corresponding to the preset data dimension in the later-stage exemplary system operation monitoring data cluster may be selected by setting the master screening parameter to be 30% and the system operation monitoring data corresponding to the other data dimension in the later-stage exemplary system operation monitoring data cluster may be selected by setting the master screening parameter to be 70%. And if the candidate anomaly analysis network is subjected to network updating operation for 100 times, 30 times of system operation monitoring data corresponding to preset data dimensions in the later-stage exemplary system operation monitoring data cluster are selected, and 70 times of system operation monitoring data corresponding to other data dimensions are selected.

Specifically, in some possible embodiments, the process of forming the preliminary exemplary data may further include the following detailed implementation:

determining an example Internet of things operation information cluster corresponding to each example data dimension, wherein the example Internet of things operation information cluster can comprise a plurality of example Internet of things operation information;

Determining identification data corresponding to each piece of operation information of the Internet of things in each piece of operation information cluster of the Internet of things to form an identification data example cluster corresponding to each piece of example data dimension, wherein the identification data can be formed based on analysis of other neural networks or formed by manual labeling;

analyzing importance characterization parameters of the identification data in each identification data exemplary cluster based on the operation duration and the operation information validity parameters corresponding to the operation information of the exemplary Internet of things corresponding to the identification data in each identification data exemplary cluster, that is, the importance characterization parameters can be fused with the corresponding operation duration and the operation information validity parameters;

analyzing the identification data screening cluster corresponding to the previous-stage exemplary data based on the importance characterization parameters of each identification data in each identification data exemplary cluster, for example, one or a specified number of multiple identification data with the largest importance characterization parameters can be extracted to serve as the identification data screening cluster corresponding to the previous-stage exemplary data;

determining screening identification data of a target number in the identification data screening cluster to determine target exemplary identification data corresponding to the early-stage exemplary data, and forming an exemplary identification data cluster corresponding to the early-stage exemplary data based on the target exemplary identification data; that is, the target exemplary identification data includes a target number of screening identification data, and a specific value of the target number is not limited, for example, the target exemplary identification data may be directly used as an exemplary identification data cluster corresponding to the early exemplary data;

And determining a front-stage exemplary system operation monitoring data cluster corresponding to the front-stage exemplary data based on other screening identification data which is not determined to be target exemplary identification data in the identification data screening cluster and the exemplary internet of things operation information cluster, for example, adding the other screening identification data which is not determined to be target exemplary identification data in the identification data screening cluster into corresponding data in the exemplary internet of things operation information cluster, so that the front-stage exemplary system operation monitoring data cluster corresponding to the front-stage exemplary data can be formed.

Specifically, in some possible embodiments, the step of analyzing the importance characterizing parameters of each piece of identification data in each piece of identification data exemplary cluster based on the operation duration and the operation information validity parameters corresponding to the operation information of the exemplary internet of things corresponding to each piece of identification data in each piece of identification data exemplary cluster may further include the following detailed implementation contents:

determining operation end time information of the operation information of the Internet of things corresponding to each piece of identification data in each piece of identification data example cluster, and determining operation information validity parameters with correlation based on the operation end time information, wherein for example, the later the operation end time information is, the larger the operation information validity parameters are; conversely, the earlier the operation end time information is, the smaller the operation information validity parameter is;

And respectively carrying out fusion calculation on the operation duration and the operation information validity parameter corresponding to the operation information of the exemplary Internet of things corresponding to each identification data in each identification data exemplary cluster, and if the operation duration and the operation information validity parameter are multiplied, obtaining the importance characterization parameter of each identification data in each identification data exemplary cluster.

Specifically, in some possible embodiments, the step of determining, in the identification data filtering cluster, a target number of filtering identification data to determine target exemplary identification data corresponding to the previous exemplary data, and forming, based on the target exemplary identification data, an exemplary identification data cluster corresponding to the previous exemplary data may further include the following detailed implementation matters:

Determining non-relevant exemplary identification data, wherein the non-relevant exemplary identification data is based on at least one of first exemplary identification data and second exemplary identification data, the first exemplary identification data belongs to relevant exemplary identification data corresponding to other exemplary data except the previous exemplary data, for example, the corresponding internet of things systems may be different, for example, the previous exemplary data may be specific to the internet of things system 1, and the first exemplary identification data may be specific to the internet of things system 2; the second exemplary identification data belongs to identification data (which can be randomly screened) in a pre-constructed identification database;

determining an exemplary identification data cluster corresponding to the early exemplary data based on the related exemplary identification data and the non-related exemplary identification data, that is, the exemplary identification data cluster may include the related exemplary identification data and the non-related exemplary identification data, that is, a gap between output data and the related exemplary identification data needs to be reduced and the output data and the non-related exemplary identification data needs to be increased in a network updating process; based on this, due to the addition of the irrelevant exemplary identification data, it can be ensured that when the data is small, the obtained early-stage exemplary data can still have enough accuracy, and the updating effect is ensured.

Specifically, in some possible embodiments, the mapping feature integration unit may include a feature internal integration unit, a feature external integration unit, a feature association integration unit, and a feature linear integration unit, based on which, the step of performing mapping feature integration operation on each running space mapping feature representation corresponding to each data dimension by using the mapping feature integration unit included in the optimized anomaly analysis network to output a running feature representation to be analyzed corresponding to the to-be-monitored internet of things system may further include the following detailed implementation content:

utilizing the feature internal integration unit to integrate each operation space mapping feature representation corresponding to each data dimension respectively so as to output an internal mapping feature representation corresponding to each data dimension, that is, to integrate each operation space mapping feature representation corresponding to each data dimension by self analysis so as to obtain an internal mapping feature representation with stronger information characterization capability, specifically, the operation space mapping feature representation corresponding to one data dimension can be multiple, that is, the operation space mapping feature representation corresponding to one data dimension can be multiple, for example, multiple devices included in the system, each device has corresponding system operation monitoring data, or can be divided according to time, that is, for multiple time periods, the operation space mapping feature representation corresponding to one data dimension can be integrated, for example, superposition or averaging operation can be performed, so as to obtain an internal mapping feature representation corresponding to one data dimension;

Utilizing the feature external integration unit to integrate each internal mapping feature representation to output an external mapping feature representation corresponding to the to-be-monitored internet of things system, namely, further enhancing the information of each data dimension, so that the information expression capability of the external mapping feature representation corresponding to the to-be-monitored internet of things system can be further improved;

performing a correlation analysis operation on the local feature representation in the external mapping feature representation by using the feature correlation integration unit to output a corresponding correlation analysis feature representation, that is, performing a correlation analysis on each part in the external mapping feature representation; illustratively, each feature representation parameter of the external mapping feature representation may be used as a local feature representation of the external mapping feature representation, so the external mapping feature representation may include a plurality of local feature representations, and thus, the associated analysis feature representation refers to a feature representation (i.e., a vector, and the feature representation parameter is a vector parameter) obtained by aggregating local feature representations in the external mapping feature representation, and capable of reflecting the correlation between the local feature representations; specifically, association analysis operation can be performed on any two or more local feature representations in the external mapping feature representations, a plurality of association local feature representations which can represent correlation among the local feature representations can be obtained after the association analysis operation, and then the association local feature representations can be aggregated to obtain association analysis feature representations corresponding to the to-be-monitored internet of things system;

And performing linear integration operation on the external mapping feature representation and the association analysis feature representation by using the feature linear integration unit so as to output an operation feature representation to be analyzed corresponding to the system of the internet of things to be monitored, wherein the feature linear integration unit can be a multi-layer perceptron.

Specifically, in some possible embodiments, the step of using the feature internal integration unit to perform an integration operation on each running space mapping feature representation corresponding to each data dimension to output an internal mapping feature representation corresponding to each data dimension, may further include the following detailed implementation content:

determining, by using the feature internal integration unit, a saliency estimation parameter respectively possessed by each of the runtime space mapping feature representations corresponding to each of the data dimensions, where, illustratively, a plurality of runtime space mapping feature representations corresponding to any one of the data dimensions may be determined, then, for any one of the plurality of runtime space mapping feature representations, performing a averaging operation on the other runtime space mapping feature representations except for the runtime space mapping feature representation to form a mean value runtime space mapping feature representation corresponding to the runtime space mapping feature representation, and analyzing, based on the mean value runtime space mapping feature representation, a saliency estimation parameter possessed by the runtime space mapping feature representation, where, for example, a number product of the mean value runtime space mapping feature representation and the runtime space mapping feature representation may be used as the saliency estimation parameter;

And performing weighted integration operation on each running space mapping feature representation corresponding to each data dimension by using the feature internal integration unit based on the corresponding saliency evaluation parameter to form an internal mapping feature representation corresponding to each data dimension, for example, for any one data dimension, the corresponding saliency evaluation parameter can be used as a weighting coefficient to perform weighted summation operation on a plurality of running space mapping feature representations corresponding to the data dimension to obtain a corresponding internal mapping feature representation.

Specifically, in some possible embodiments, the step of integrating, by using the feature external integration unit, each internal mapping feature representation to output an external mapping feature representation corresponding to the to-be-monitored internet of things system may further include the following detailed implementation contents:

determining, by the feature external integration unit, a saliency estimation parameter corresponding to each of the internal mapping feature representations, by way of example, for any one internal mapping feature representation, performing a averaging operation on each other internal mapping feature representation other than the internal mapping feature representation to form a corresponding mean internal mapping feature representation, and determining, based on the mean internal mapping feature representation, a saliency estimation parameter corresponding to the internal mapping feature representation, e.g., a number product between the mean internal mapping feature representation and the internal mapping feature representation may be used as the saliency estimation parameter corresponding to the internal mapping feature representation;

And carrying out weighted integration operation on each internal mapping feature representation based on the corresponding significance evaluation parameters by utilizing the feature external integration unit, and carrying out weighted summation calculation to form the external mapping feature representation corresponding to the Internet of things system to be monitored.

Specifically, in some possible embodiments, the step of performing, by using the feature association integration unit, an association analysis operation on the local feature representation in the external mapping feature representation to output a corresponding association analysis feature representation may further include the following detailed implementation content:

the feature association integration unit is utilized to perform segmentation and combination operation on feature representation parameters in the external mapping feature representation to form a plurality of local feature representations corresponding to the external mapping feature representation, wherein the sizes of every two local feature representations in the plurality of local feature representations are consistent, for example, the number of rows and the number of columns of the included feature representation parameters are consistent, for example, 1 row and 1 column, 2 rows and 2 columns, 3 rows and 3 columns, 8 rows and 8 columns and the like;

weighting each local feature representation in the multiple local feature representations by using each first parameter included in a first parameter matrix included in the feature association and integration unit, and performing superposition operation on the weighted local feature representations by using each second parameter included in a second parameter matrix included in the feature association and integration unit to form corresponding transformed local feature representations, namely obtaining multiple corresponding transformed local feature representations, wherein parameters for the segmentation and combination operation, the first parameter matrix and the second parameter matrix can be formed in a corresponding neural network updating process;

Respectively carrying out dot product calculation on each local feature representation and the corresponding transformation local feature representation to obtain a plurality of corresponding dot product calculation values;

and respectively carrying out dot product calculation on at least two dot product calculation values or every two dot product calculation values in the plurality of dot product calculation values to obtain at least one target dot product calculation value, and finally, determining a corresponding association analysis characteristic representation based on the at least one target dot product calculation value, for example, combining the at least one target dot product calculation value together to form a corresponding association analysis characteristic representation, wherein the association analysis characteristic representation comprises at least one target dot product calculation value combination.

Specifically, in some possible embodiments, the feature linear integration unit may include a first feature linear integration unit and a second feature linear integration unit, based on which, the step of performing linear integration operation on the external mapping feature representation and the association analysis feature representation by using the feature linear integration unit to output a running feature representation to be analyzed corresponding to the system of the internet of things to be monitored may further include the following detailed implementation contents:

Performing a dimension transformation operation on the associated analysis feature representation with the first feature linear integration unit to form a dimension transformation feature representation having the same dimension as the dimension of the external mapping feature representation, and performing a cascading combination operation on the dimension transformation feature representation and the external mapping feature representation to form a corresponding cascading combination feature representation, such as { the dimension transformation feature representation, the external mapping feature representation }; mapping the cascade combination characteristic representation into an operation characteristic representation to be analyzed corresponding to the system of the internet of things to be monitored by utilizing the second characteristic linear integration unit; the first characteristic linear integration unit may be composed of a multi-layer fully connected neural network, for example.

Specifically, in some possible embodiments, the step of using the optimized anomaly analysis network to match the system operation identification data corresponding to the to-be-analyzed operation feature representation in the to-be-identified data cluster corresponding to the preset data dimension to form the system operation identification data cluster corresponding to the to-be-monitored internet of things system may further include the following detailed implementation contents:

For each piece of undetermined identification data in the undetermined identification data cluster corresponding to the preset data dimension, performing feature space mapping operation on the undetermined identification data, for example, encoding through an encoding network to form an identification mapping feature representation corresponding to the undetermined identification data, and performing matching degree calculation on the identification mapping feature representation and the to-be-analyzed operation feature representation, for example, calculating cosine similarity, so that feature representation matching parameters corresponding to the undetermined identification data can be obtained;

based on the characteristic representation matching parameters, matching the system operation identification data corresponding to the operation characteristic representation to be analyzed in the undetermined identification data cluster corresponding to the preset data dimension to form a system operation identification data cluster corresponding to the system of the Internet of things to be monitored; for example, the feature representation matching parameter corresponding to the system operation identification data may be greater than or equal to a pre-configured reference feature representation matching parameter, where the reference feature representation matching parameter may be configured according to requirements.

With reference to fig. 3, the embodiment of the invention also provides an internet of things abnormality monitoring device based on data mining, which can be applied to the internet of things abnormality monitoring system based on data mining. The data mining-based internet of things anomaly monitoring device may include:

The system operation monitoring system comprises an operation monitoring data determining module, a monitoring data processing module and a monitoring data processing module, wherein the operation monitoring data determining module is used for determining a system operation monitoring data cluster corresponding to an Internet of things system to be monitored, and system operation monitoring data in the system operation monitoring data cluster comprises a system operation log;

the analysis network determining module is used for determining an optimized abnormal analysis network corresponding to the system operation monitoring data cluster;

the operation anomaly analysis module is used for performing operation anomaly analysis operation on the system operation monitoring data cluster by utilizing the optimized anomaly analysis network so as to analyze a system operation identification data cluster corresponding to the system of the internet of things to be monitored, wherein the system operation identification data cluster comprises a plurality of system operation identification data, and the plurality of system operation identification data are used for reflecting operation anomaly prediction information of the system of the internet of things to be monitored on a plurality of operation monitoring layers.

In summary, according to the data mining-based method and system for monitoring the anomaly of the internet of things, the system operation monitoring data cluster corresponding to the system of the internet of things to be monitored can be determined; determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster; and carrying out operation anomaly analysis operation on the system operation monitoring data cluster by utilizing the optimized anomaly analysis network so as to analyze a system operation identification data cluster corresponding to the system of the Internet of things to be monitored. Based on the foregoing, because the analyzed system operation identification data cluster includes a plurality of system operation identification data, the content for representing the abnormal operation is richer, so that the reliability of monitoring the abnormal operation of the internet of things can be improved to a certain extent, and the problem of low reliability of monitoring the abnormal operation of the internet of things in the prior art is solved.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. The method for monitoring the abnormality of the Internet of things based on data mining is characterized by comprising the following steps of:

2. The method for monitoring the anomaly of the internet of things based on data mining according to claim 1, wherein the system operation monitoring data cluster comprises system operation monitoring data corresponding to at least one data dimension other than a preset data dimension; the optimized anomaly analysis network is formed by performing network updating operation on candidate anomaly analysis networks based on later-stage exemplary data, the later-stage exemplary data comprises later-stage exemplary system operation monitoring data clusters and exemplary identification data corresponding to preset data dimensions, the later-stage exemplary system operation monitoring data clusters comprise system operation monitoring data corresponding to the preset data dimensions, the candidate anomaly analysis network is formed by performing network updating operation on original anomaly analysis networks based on earlier-stage exemplary data, and the earlier-stage exemplary data comprises earlier-stage exemplary system operation monitoring data clusters corresponding to a plurality of exemplary data dimensions and corresponding exemplary identification data clusters;

3. The method for monitoring the anomaly of the internet of things based on data mining according to claim 2, wherein the forming process of the post-exemplary data comprises the following steps:

4. The data mining-based internet of things anomaly monitoring method of claim 2, wherein the network update process of the candidate anomaly analysis network comprises:

5. The method for monitoring the anomaly of the internet of things based on data mining according to claim 2, wherein the forming process of the pre-exemplary data comprises the following steps:

6. The method for monitoring abnormal internet of things based on data mining according to claim 5, wherein the step of determining a target number of screening identification data in the identification data screening cluster to determine target exemplary identification data corresponding to the pre-exemplary data, and forming an exemplary identification data cluster corresponding to the pre-exemplary data based on the target exemplary identification data, comprises:

7. The data mining-based internet of things anomaly monitoring method of claim 2, wherein the mapping feature integration unit comprises a feature internal integration unit, a feature external integration unit, a feature association integration unit and a feature linear integration unit;

8. The method for monitoring abnormal internet of things based on data mining according to claim 7, wherein the step of integrating each of the running space mapping feature representations corresponding to each of the data dimensions by using the feature internal integration unit to output an internal mapping feature representation corresponding to each of the data dimensions comprises:

9. The method for monitoring anomalies of internet of things based on data mining according to any one of claims 1-8, wherein the step of determining an optimized anomaly analysis network corresponding to the system operation monitoring data cluster includes:

10. The data mining-based internet of things anomaly monitoring system is characterized by comprising a processor and a memory, wherein the memory is used for storing a computer program, and the processor is used for executing the computer program to realize the data mining-based internet of things anomaly monitoring method according to any one of claims 1-9.