CN117932455A - Internet of things asset identification method and system based on neural network - Google Patents

Abstract

The invention relates to an internet of things asset identification method and system based on a neural network, wherein the method comprises the following steps: identifying and acquiring image data and flow data of internet of things equipment in an internet of things network; encoding and decoding the image data and the flow data through an automatic encoder of the initial asset identification model to obtain image characteristics and flow characteristics; carrying out fusion processing on the image characteristics and the flow characteristics to generate a fused integrated representation; predicting the integrated representation through an initial asset identification model to obtain a predicted value corresponding to the integrated representation; constructing a loss function of an initial asset identification model; training an initial asset identification model according to the training set, the labels of the training set and the loss function until a trained asset identification model is obtained; and inputting the image to be identified and the flow to be identified of the equipment of the internet of things to the asset identification model to obtain the asset class of the equipment of the internet of things to be identified. The method and the device can improve the asset identification efficiency of the Internet of things equipment.

Description

Internet of things asset identification method and system based on neural network

Technical Field

The invention relates to the technical field of data identification, in particular to an internet of things asset identification method, system, electronic equipment and non-transitory computer readable storage medium based on a neural network.

Background

Today, the number of internet of things devices worldwide has increased dramatically, and there is a lack of effective supervision of the number, type, brand, and operating system of these devices. Asset identification technology has been developed, and various assets in a monitoring picture are automatically identified by using computer vision, so that intelligent management of the Internet of things equipment is realized.

However, the existing asset identification technology mainly monitors network traffic to extract characteristic information and compares the characteristic information with sample tag data to complete equipment identification. With the expansion of the scale of the Internet of things, equipment is continuously increased or replaced, the identification efficiency and accuracy of the method are low, the comparison operation of processing large-scale network traffic and sample tag data becomes time-consuming, or the equipment is repeatedly and re-identified due to low identification accuracy, so that the timely and accurate management of the equipment of the Internet of things is affected. This drawback makes it difficult for supervisory personnel to cope with rapidly changing internet of things environments, limiting the effective monitoring and management of devices.

Disclosure of Invention

Aiming at the technical problems in the prior art, the invention provides an internet of things asset identification method, an internet of things asset identification system, electronic equipment and a non-transitory computer readable storage medium, which can improve the asset identification efficiency of internet of things equipment.

The technical scheme for solving the technical problems is as follows:

The invention provides an internet of things asset identification method based on a neural network, which comprises the following steps:

identifying and acquiring image data and flow data of internet of things equipment in an internet of things network;

Encoding and decoding the image data and the flow data through an automatic encoder of an initial asset identification model to obtain image characteristics and flow characteristics;

Carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

Predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

Constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

training the initial asset identification model according to a training set comprising a plurality of image data and a plurality of flow data, a label of the training set and the loss function until a trained asset identification model is obtained;

and inputting the image to be identified and the flow to be identified of the equipment of the internet of things to the asset identification model to obtain the asset type of the equipment of the internet of things to be identified.

Optionally, the encoding and decoding process is performed on the image data and the flow data by the automatic encoder of the initial asset identification model to obtain image features and flow features, including:

invoking an encoding unit and a decoding unit of the automatic encoder;

Mapping the image data and the flow data to a potential space through the encoding unit to obtain potential characteristics of the image data and the flow data;

And mapping the potential features back to the original data space of the image data and the flow data through the decoding unit so as to reconstruct the image data and the flow data and obtain the corresponding image features and flow features.

Optionally, the first loss function of the automatic encoder for performing codec processing on the image data and the flow data is:

；

Wherein AE is the automatic encoder described above, Is the first parameter of the automatic encoder,/>Is the image data,/>Is decoded image data, i.e. the image features,/>Is an image coding unit of the coding units,/>Is an image decoding unit of the decoding units,/>A first regularization term of the first parameter, alpha being a first weight;

；

Wherein AE is the automatic encoder described above, Is a second parameter of the automatic encoder,/>Is the traffic data,/>Is the decoded traffic data, i.e. the traffic characteristics,/>Is the flow coding unit in the coding units,/>Is a traffic decoding unit of the decoding units,/>Is a second regularization term for the second parameter.

Optionally, the constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value includes:

Constructing a second loss function according to the integrated representation, the predicted value and the label of the predicted value; the expression of the second loss function is:

；

wherein, Is the predicted value, Y is a label of the predicted value,/>Is the integrated representation, lambda, gamma,/>Delta is the second weight, the third weight, the fourth weight and the fifth weight, respectively.

Optionally, the predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation includes:

Obtaining at least one of the integrated representations;

Inputting at least one of the integrated representations into the initial asset identification model for forward propagation, processing through a plurality of intermediate layers until reaching an output layer;

And obtaining a first characteristic output by the output layer, and processing the first characteristic through an activation function to obtain the predicted value.

Optionally, the expression of the predicted value is:

；

wherein, For the predicted value,/>For the integrated representation, the output of the fully connected layer, i.e., the first feature,/>, via the initial asset identification modelFor the weight of the output layer,/>Is the bias of the output layer.

Optionally, the identifying and acquiring the image data of the internet of things device in the internet of things network includes:

Acquiring an original image of the Internet of things equipment through image acquisition equipment;

Acquiring a GAN generator, and inputting the original image to the GAN generator;

And processing the original image through the GAN generator to obtain a simulation image similar to the original image, and taking the original image and the simulation image as the image data.

Optionally, before said predicting said integrated representation by said initial asset identification model, further comprising:

Clustering a plurality of integrated representations by adopting a clustering algorithm to obtain a clustering label corresponding to each integrated representation; the cluster label is used for representing a cluster corresponding to the integrated representation;

Partitioning at least one of the integrated representations having the same cluster tag into the same panel;

and ordering the plurality of integrated representations in the order of the groups so that the initial asset identification model preferentially processes the integrated representations belonging to the same group.

The invention also provides an internet of things asset identification system based on the neural network, which comprises:

The data identification module is used for identifying and acquiring image data and flow data of the Internet of things equipment in the Internet of things network;

the feature extraction module is used for carrying out encoding and decoding processing on the image data and the flow data through an automatic encoder of the initial asset identification model to obtain image features and flow features;

the feature fusion module is used for carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

The model prediction module is used for predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

The loss construction module is used for constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

The model training module is used for training the initial asset identification model according to a training set containing a plurality of image data and a plurality of flow data, the labels of the training set and the loss function until a trained asset identification model is obtained;

The asset identification module is used for inputting an image to be identified and flow to be identified of the equipment of the internet of things to be identified into the asset identification model to obtain the asset category of the equipment of the internet of things to be identified.

Optionally, the feature extraction module is further configured to:

invoking an encoding unit and a decoding unit of the automatic encoder;

Mapping the image data and the flow data to a potential space through the encoding unit to obtain potential characteristics of the image data and the flow data;

And mapping the potential features back to the original data space of the image data and the flow data through the decoding unit so as to reconstruct the image data and the flow data and obtain the corresponding image features and flow features.

In addition, to achieve the above object, the present invention also proposes an electronic device including: a memory for storing a computer software program; and the processor is used for reading and executing the computer software program so as to realize the internet of things asset identification method based on the neural network.

In addition, in order to achieve the above object, the present invention also proposes a non-transitory computer readable storage medium, in which a computer software program is stored, which when executed by a processor, implements a neural network-based asset identification method as described above.

The beneficial effects of the invention are as follows:

(1) Compared with the existing asset identification technology, the method and the device can fully utilize the information of different data sources by integrating a small amount of images and flow data of the equipment to be identified, avoid the defect of excessively relying on network flow monitoring, and improve the efficiency and accuracy of equipment identification.

(2) Compared with the existing asset identification technology, the invention utilizes the image and flow data to perform unsupervised learning by introducing the automatic encoder, so that the useful features in the data are more effectively learned, and the intelligent management effect on the equipment is improved.

(3) Compared with the existing asset identification technology, the method encourages models to learn meaningful potential space structures by integrating the expressed clustering loss items, adapts to the rapidly-changing environment of the Internet of things better, and improves the coping capability of equipment changes.

(4) Compared with the prior asset identification technology, the invention uses gradient regularization and automatic encoder parametersRegularization improves training stability of the model, prevents gradient-related problems, effectively controls complexity of the model, and facilitates rapid processing of large-scale data.

(5) Compared with the existing asset identification technology, the method and the system adopt comprehensive loss functions, balance a plurality of training targets, help to improve the performance of the model in various aspects, and better adapt to the change of the environment of the Internet of things, thereby improving the problem of effective monitoring and management of the equipment by supervisory personnel.

Drawings

FIG. 1 is a scene diagram of an asset identification method of the Internet of things based on a neural network;

FIG. 2 is a flow chart of an asset identification method of the Internet of things based on the neural network;

Fig. 3 is a schematic structural diagram of an internet of things asset identification system based on a neural network provided by the invention;

Fig. 4 is a schematic hardware structure of one possible electronic device according to the present invention;

Fig. 5 is a schematic hardware structure of a possible computer readable storage medium according to the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the description of the present invention, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

In the description of the present invention, the term "for example" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "for example" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the invention. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the invention with unnecessary detail. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

Referring to fig. 1, fig. 1 is a schematic diagram of an asset identification method of internet of things based on a neural network. As shown in fig. 1, the terminal and the server are connected through a network, for example, a wired or wireless network connection. The terminal may include, but is not limited to, mobile terminals such as mobile phones and tablets, and fixed terminals such as computers, inquiry machines and advertising machines, where applications of various network platforms are installed. The server provides various business services for the user, including a service push server, a user recommendation server and the like.

It should be noted that, the scene graph of the internet of things asset identification method shown in fig. 1 is only an example, and the terminal, the server and the application scenario described in the embodiment of the present invention are for more clearly describing the technical solution of the embodiment of the present invention, and do not generate a limitation on the technical solution provided by the embodiment of the present invention, and as a person of ordinary skill in the art knows, along with the evolution of the system and the appearance of a new service scenario, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

Wherein the terminal may be configured to:

identifying and acquiring image data and flow data of internet of things equipment in an internet of things network;

Encoding and decoding the image data and the flow data through an automatic encoder of an initial asset identification model to obtain image characteristics and flow characteristics;

Carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

Predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

Constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

training the initial asset identification model according to a training set comprising a plurality of image data and a plurality of flow data, a label of the training set and the loss function until a trained asset identification model is obtained;

and inputting the image to be identified and the flow to be identified of the equipment of the internet of things to the asset identification model to obtain the asset type of the equipment of the internet of things to be identified.

Referring to fig. 2, a flowchart of an internet of things asset identification method based on a neural network is provided, which includes the following steps:

Step 201, identifying and acquiring image data and flow data of internet of things equipment in an internet of things network.

In some embodiments, image data of the internet of things device may be acquired through a network monitoring camera, a sensor, or the like. For example, the image data may include information about the appearance, identity, status, etc. of the device.

In some embodiments, network traffic data of the internet of things device may be obtained through network monitoring, data packet capturing, and the like. Traffic data contains information about the communication mode of the device, network activity, data transmission, etc.

For example, the traffic data may be json format data such as {"ip":"6","prot":2,"vlan_id":8,"src_ip":"192.168.1.10","dst_ip":"192.168.2.20","src_port":8080,"dst_port":443,"c2s": ...}.

Alternatively, step 201 may comprise the steps of:

Acquiring an original image of the Internet of things equipment through image acquisition equipment;

Acquiring a GAN generator, and inputting the original image to the GAN generator;

And processing the original image through the GAN generator to obtain a simulation image similar to the original image, and taking the original image and the simulation image as the image data.

In some embodiments, an image capture device, such as a monitoring camera or sensor, may be used to obtain an original image from an internet of things device, which may capture information of the appearance, status, identification, etc. of the internet of things device.

In some embodiments, a generated countermeasure network (GAN generator) that has been trained may be obtained, the GAN being a deep learning model that can learn to generate new samples that are similar to a given data distribution.

In some embodiments, the acquired original image of the internet of things device may be input as an input to a previously acquired GAN generator to generate a simulated image that approximates the original image using a generator model.

In some embodiments, the input raw image may be processed by a GAN generator that generates a simulated image in a learned pattern, the GAN generator targeting that the simulated image is visually similar to the raw image, but is not required to be completely consistent.

In some embodiments, the original image and the generated simulated image may be taken as final image data. The combination of the two provides more data samples, which helps to train neural network models, especially automatic encoders for image feature extraction and integration.

In some embodiments, the loss function for training the GAN generator may be expressed as:

；

Wherein G is a generator and D is a discriminator; Representing the logarithm of the probability (discrimination as true data) that the discriminator D outputs for the true data x, the generator expects this term to be maximized to spoof the discriminator so that it considers the generated spurious data to be true;

Representing the logarithm of the probability (discriminating as fake data) output by the discriminator D for fake data G (z) generated by the generator, the discriminator expects this term to be maximized to correctly identify the generated fake data.

The goal of GAN is to train the generator G and the arbiter D through the game so that the generator generates realistic data while the arbiter accurately distinguishes between the realistic data and the generated counterfeit data.

At the level of the mathematical interpretation,Representing the expectation of all real data, wherein log d (x) is the logarithm of the probability that the arbiter outputs as real data;

Representing expectations for all of the generator generated data, where Is the logarithm of the probability that the arbiter outputs as spurious data.

Through the mode, the image data of the Internet of things equipment are expanded, so that more samples are available for the model in learning and extracting the image features, the accuracy of identification and initial asset information prediction of the Internet of things equipment is improved, more diversified and rich image data can be generated by using GAN, and the generalization capability of the model is enhanced.

And 202, encoding and decoding the image data and the flow data through an automatic encoder of an initial asset identification model to obtain image characteristics and flow characteristics.

Optionally, step 202 may further include the steps of:

invoking an encoding unit and a decoding unit of the automatic encoder;

Mapping the image data and the flow data to a potential space through the encoding unit to obtain potential characteristics of the image data and the flow data;

And mapping the potential features back to the original data space of the image data and the flow data through the decoding unit so as to reconstruct the image data and the flow data and obtain the corresponding image features and flow features.

In some embodiments, the automatic encoder may include an encoding unit that in turn includes an image encoding unit and a traffic encoding unit, and a decoding unit that in turn includes an image decoding unit and a traffic decoding unit.

In this process, the automatic encoder is used for feature extraction and reconstruction, compression and restoration of image data and flow data is achieved through the representation of the underlying space, which will be used for subsequent integrated representation and further processing.

In some embodiments, the first loss function of the automatic encoder to codec the image data and the traffic data is:

；

Wherein AE is the automatic encoder described above, Is the first parameter of the automatic encoder,/>Is the image data,/>Is decoded image data, i.e. the image features,/>Is an image coding unit of the coding units,/>Is an image decoding unit of the decoding units,/>A first regularization term of the first parameter, alpha being a first weight;

；

Wherein AE is the automatic encoder described above, Is a second parameter of the automatic encoder,/>Is the traffic data,/>Is the decoded traffic data, i.e. the traffic characteristics,/>Is the flow coding unit in the coding units,/>Is a traffic decoding unit of the decoding units,/>Is a second regularization term for the second parameter.

The automatic encoder of the present invention includes an image encoder for processing image data to obtain image features, and a text encoder for processing flow data to obtain flow features. Therefore, the data can be encoded by taking the characteristics of different types of data into consideration through an initial asset identification model, and the characterization capacity and the prediction accuracy of the model are improved.

It should also be noted that,The square of the euclidean norm is expressed for measuring the difference between the original image and the reconstructed image and also for measuring the difference between the original flow data and the reconstructed flow data. It will be appreciated that the goal of the above two first loss functions is to minimize the difference between the original data and the reconstructed data and to prevent overfitting by regularization terms. This helps the auto encoder learn the feature representation that efficiently encodes and decodes image data and traffic data.

And 203, performing fusion processing on the image features and the flow features to generate a fused integrated representation.

Wherein the image features areFlow is characterized by/>. In some embodiments, image features/>, may be combined using a specialized integration layerAnd flow characteristics/>The fusion is carried out, and the design of an integration layer can involve the modes of weighted summation, splicing, element-by-element multiplication and the like. Wherein the output of the integration layer is the integrated representation/>This representation captures the combined information of the image and flow characteristics.

And 204, predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation.

In some embodiments, the initial asset identification model may utilize an integrated representation of the inputsAnd predicting to obtain a corresponding predicted value. For example, the predicted value may contain identification information, category, status, etc. about the internet of things device.

Optionally, before step 204, the method of the present application further comprises:

Clustering a plurality of integrated representations by adopting a clustering algorithm to obtain a clustering label corresponding to each integrated representation; the cluster label is used for representing a cluster corresponding to the integrated representation;

Partitioning at least one of the integrated representations having the same cluster tag into the same panel;

and ordering the plurality of integrated representations in the order of the groups so that the initial asset identification model preferentially processes the integrated representations belonging to the same group.

In some embodiments, the multiple integrated representations may be clustered using a clustering algorithm, such as K-Means. Each integrated representation will be assigned to a cluster, resulting in a corresponding cluster label.

In some embodiments, each integrated representation may be assigned a cluster label that reflects the similarity or association between the integrated representations to which cluster the integrated representation belongs.

In some embodiments, the integrated representations with the same cluster labels may be partitioned into the same group. This means that the integrated representations in the same panel have similar features within the cluster space.

In some embodiments, the multiple integrated representations may be ordered in the order in which they belong to the group. This helps to ensure that the integrated representations of the same panel are continuous when processed.

In some embodiments, the ranked integrated representation may be input to an initial asset identification model. Since the integrated representations with the same cluster labels are partitioned into the same group and ordered in order, the initial asset identification model will prioritize the integrated representations of the same group.

By means of the method, the integrated representations can be clustered and grouped, the integrated representations divided into the same group are arranged according to the sequence of the integrated representations, the processing method is beneficial to improving processing efficiency, and the initial asset identification model can be used for processing similar integrated representations more consistently, so that the method is better suitable for rapid changes of the environment of the Internet of things.

Optionally, step 204 may include:

Obtaining at least one of the integrated representations;

Inputting at least one of the integrated representations into the initial asset identification model for forward propagation, processing through a plurality of intermediate layers until reaching an output layer;

And obtaining a first characteristic output by the output layer, and processing the first characteristic through an activation function to obtain the predicted value.

In some embodiments, the expression of the predictor is:

；

wherein, For the predicted value,/>For the integrated representation, the output of the fully connected layer, i.e., the first feature,/>, via the initial asset identification modelFor the weight of the output layer,/>Is the bias of the output layer.

In some embodiments, at least one integrated representation may be input into the initial asset identification model for forward propagation, with the integrated representation progressively extracting and converting features through multiple intermediate layers, and ultimately reaching the output layer.

In particular implementation, the first feature of the output layerIs obtained by integrating the features obtained by the full connection layer treatment in the forward propagation process, and can be applied to the first features/>Applying the activation function to obtain a predicted value/>The activation function may employ a sigmoid function for mapping the output value to a range of (0, 1), representing a probability or some confidence.

By the method, the integrated representation can be processed through the model to obtain the corresponding predicted value, and the predicted value can be used for judging the attribute, the state or other related information of the Internet of things equipment.

Step 205, constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value.

Optionally, step 205 may include the steps of:

Constructing a second loss function according to the integrated representation, the predicted value and the label of the predicted value; the expression of the second loss function is:

；

wherein, Is the predicted value, Y is a label of the predicted value,/>Is the integrated representation, lambda, gamma,/>Delta is the second weight, the third weight, the fourth weight and the fifth weight, respectively.

In a specific implementation of the present invention,log(/>) Is cross entropy loss for measuring predicted value/>The difference from tag Y, cross entropy loss, is a sort of task loss function that measures the performance of the model by comparing the predicted value to the distribution of true tags.

Is the sum of features of the integrated representation, where λ is the second weight, and feature information of the integrated representation is introduced, which helps the model to better understand the features in the integrated representation.

Is the sum of absolute values of the integrated representation features with respect to the gradient of the integrated representation, where/>Is a third weight that is used to measure the impact of the integrated representation features on the integrated representation, helping the model to focus more on features important to the task.

Is a regularization term for the auto-encoder parameters, where β and δ are the fourth and fifth weights, respectively, to help control the magnitude of the auto-encoder parameters, avoiding overfitting.

Through each item, the second loss function can consider all aspects of the integrated representation, the predicted value and the automatic encoder, so that a plurality of targets can be more comprehensively optimized when the model is trained, and the accuracy and the robustness of the identification and the management of the Internet of things equipment are improved.

And step 206, training the initial asset identification model according to a training set comprising a plurality of image data and a plurality of flow data, the labels of the training set and the loss function until a trained asset identification model is obtained.

In some embodiments, a training set may be obtained that includes a plurality of image data and flow data, each sample having a corresponding tag that identifies an attribute, state, or other relevant information of the internet of things device.

In some embodiments, a second defined loss function may be used to construct a comprehensive loss function in combination with the codec loss of image data, flow data, and other regularization terms, where the loss function is used to measure the performance of the model during training, and optimize the parameters of the initial asset identification model to obtain a trained asset identification model to adapt to the identification task of the internet of things device.

In some embodiments, parameters of the initial asset identification model may be initialized, including the network structure of the integrated representation, parameters of the auto encoder, and weights and biases of other network layers. In some embodiments, the parameters of the initial asset identification model may be optimized by a back propagation algorithm with the image data and flow data of the training set, and corresponding labels, with the objective of minimizing the defined loss function. In some embodiments, multiple iterations may be performed on the training set, each of which updates the parameters of the initial asset identification model, gradually improving performance on the training data by continually iterating and optimizing.

In some embodiments, training is stopped when the performance of the initial asset identification model on the training set converges to a satisfactory level. At this time, the model has learned the characteristic representation of the image data, the flow data, and can make predictions from the integrated representation. The method comprises the steps of obtaining a trained asset identification model, generating an integrated representation according to image data and flow data of the Internet of things equipment, and obtaining a predicted value through forward propagation. The model has better generalization capability and can be used for recognition and management tasks of unseen Internet of things equipment.

Through the method, the trained asset identification model can be better adapted to the image and flow data in the environment of the Internet of things, and the identification accuracy and the overall management effect of equipment are improved.

Step 207, inputting an image to be identified and a flow to be identified of the internet of things equipment to the asset identification model to obtain an asset class of the internet of things equipment to be identified.

In some embodiments, asset classes may be used to represent information about the attributes, status, type, etc. of the device, depending on the tags used in model training. For example, asset classes may characterize the type, state, function, location, security level, purpose, etc. of the internet of things device. For example, device type categories may include cameras, sensors, smart lights, smart sockets, and the like. The device status categories may include online, offline, abnormal, normal, etc. The device function categories may include monitoring, control, communication, data collection, and the like. The device location categories may include indoor, outdoor, floor, room, etc. The device security level categories may include low risk, medium risk, high risk, and the like. The device usage categories may include home automation, industrial monitoring, medical devices, and the like.

In particular, each sample in the training dataset will have a corresponding tag describing the asset class to which the sample belongs. During training, the model adjusts the parameters by minimizing the loss function so that the output of the model is as close as possible to the true class represented by the tag.

In the trained asset identification model, probability values output by the asset identification model represent probability distributions for each asset class. Typically, the category with the highest probability may be selected as the final asset class prediction result. For example, if the probability distribution output by the asset identification model is [0.2, 0.7, 0.1], and if the probability distribution corresponds to three device types of cameras, sensors and intelligent lamps, it may be determined that the internet of things device belongs to the second type with the largest probability value, that is, after asset identification, it is determined that the device type of the internet of things device is the sensor.

In practical application, a threshold can be set according to requirements to judge whether the probability of model output is reliable enough or not. If the probability of a class is below a threshold, then the model may be considered to be less reliable for predicting the class, requiring further processing or manual confirmation. For example only, assuming a probability threshold of 0.7, if the probability distribution of the output is [0.3, 0.6, 0.1], it can be seen that all three probability values are below the probability threshold, which is equivalent to the fact that the output of the current asset identification model is invalid, and the model should be re-identified or re-trained.

In summary, the mapping relation between the probability value output by the model and the asset class is established through the label information in the training process, and meanwhile, the final asset class needs to be determined according to the size of the probability value and a set threshold value in practical application.

It should be further noted that the asset identification model of the present invention can complete asset identification based on only a small amount of images to be identified and traffic to be identified, and this advantage makes the scheme more feasible and practical in practical application. By fully utilizing the characteristic learning capability of the deep learning model, a better recognition effect can be obtained even under the condition of a smaller data volume. The method has important significance for some scenes with limited resources or scarce data, can reduce the data acquisition and training cost, and improves the deployment efficiency of the system.

Referring to fig. 3, fig. 3 is a schematic structural diagram of an internet of things asset identification system based on a neural network according to the present invention.

As shown in fig. 3, an internet of things asset identification system based on a neural network according to an embodiment of the present invention includes:

The data identifying module 301 is configured to identify and obtain image data and flow data of an internet of things device in an internet of things network;

The feature extraction module 302 is configured to perform encoding and decoding processing on the image data and the flow data through an automatic encoder of the initial asset identification model, so as to obtain image features and flow features;

The feature fusion module 303 is configured to perform fusion processing on the image feature and the flow feature, and generate a fused integrated representation;

The model prediction module 304 is configured to predict the integrated representation through the initial asset identification model, so as to obtain a predicted value corresponding to the integrated representation;

a loss construction module 305, configured to construct a loss function of the initial asset identification model according to the integrated representation, the predicted value, and the label of the predicted value;

A model training module 306, configured to train the initial asset identification model according to a training set including a plurality of the image data and a plurality of the flow data, a tag of the training set, and the loss function, until a trained asset identification model is obtained;

the asset identification module 307 is configured to input an image to be identified and a flow to be identified of the internet of things device to the asset identification model, so as to obtain an asset class of the internet of things device to be identified.

In some embodiments, the feature extraction module 302 is further to:

invoking an encoding unit and a decoding unit of the automatic encoder;

Mapping the image data and the flow data to a potential space through the encoding unit to obtain potential characteristics of the image data and the flow data;

And mapping the potential features back to the original data space of the image data and the flow data through the decoding unit so as to reconstruct the image data and the flow data and obtain the corresponding image features and flow features.

Referring to fig. 4, fig. 4 is a schematic diagram of an embodiment of an electronic device according to an embodiment of the invention. As shown in fig. 4, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420, wherein the processor 420 executes the computer program 411 to implement the following steps:

identifying and acquiring image data and flow data of internet of things equipment in an internet of things network;

Encoding and decoding the image data and the flow data through an automatic encoder of an initial asset identification model to obtain image characteristics and flow characteristics;

Carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

Predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

Constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

training the initial asset identification model according to a training set comprising a plurality of image data and a plurality of flow data, a label of the training set and the loss function until a trained asset identification model is obtained;

and inputting the image to be identified and the flow to be identified of the equipment of the internet of things to the asset identification model to obtain the asset type of the equipment of the internet of things to be identified.

Referring to fig. 5, fig. 5 is a schematic diagram of an embodiment of a computer readable storage medium according to an embodiment of the invention. As shown in fig. 5, the present embodiment provides a computer-readable storage medium 500 having stored thereon a computer program 411, which computer program 411, when executed by a processor, performs the steps of:

identifying and acquiring image data and flow data of internet of things equipment in an internet of things network;

Encoding and decoding the image data and the flow data through an automatic encoder of an initial asset identification model to obtain image characteristics and flow characteristics;

Carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

Predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

Constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

training the initial asset identification model according to a training set comprising a plurality of image data and a plurality of flow data, a label of the training set and the loss function until a trained asset identification model is obtained;

and inputting the image to be identified and the flow to be identified of the equipment of the internet of things to the asset identification model to obtain the asset type of the equipment of the internet of things to be identified.

In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (10)

1. An internet of things asset identification method based on a neural network, which is characterized by comprising the following steps:

identifying and acquiring image data and flow data of internet of things equipment in an internet of things network;

Encoding and decoding the image data and the flow data through an automatic encoder of an initial asset identification model to obtain image characteristics and flow characteristics;

Carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

Predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

Constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

training the initial asset identification model according to a training set comprising a plurality of image data and a plurality of flow data, a label of the training set and the loss function until a trained asset identification model is obtained;

and inputting the image to be identified and the flow to be identified of the equipment of the internet of things to the asset identification model to obtain the asset type of the equipment of the internet of things to be identified.

2. The neural network-based internet of things asset identification method according to claim 1, wherein the encoding and decoding the image data and the flow data by the automatic encoder of the initial asset identification model to obtain an image feature and a flow feature, comprises:

invoking an encoding unit and a decoding unit of the automatic encoder;

Mapping the image data and the flow data to a potential space through the encoding unit to obtain potential characteristics of the image data and the flow data;

And mapping the potential features back to the original data space of the image data and the flow data through the decoding unit so as to reconstruct the image data and the flow data and obtain the corresponding image features and flow features.

3. The neural network-based internet of things asset identification method of claim 2, wherein the first loss function of the automatic encoder for codec processing of the image data and the traffic data is:

；

Wherein AE is the automatic encoder described above, Is the first parameter of the automatic encoder,/>Is the image data,/>Is the decoded image data, i.e. the image features,Is an image coding unit of the coding units,/>Is an image decoding unit of the decoding units,A first regularization term of the first parameter, alpha being a first weight;

；

Wherein AE is the automatic encoder described above, Is a second parameter of the automatic encoder,/>Is the traffic data,/>Is the decoded traffic data, i.e. the traffic characteristics,Is the flow coding unit in the coding units,/>Is a traffic decoding unit of the decoding units,Is a second regularization term for the second parameter.

4. The neural network-based internet of things asset identification method of claim 3, wherein constructing the loss function of the initial asset identification model from the integrated representation, the predicted value, and the label of the predicted value comprises:

Constructing a second loss function according to the integrated representation, the predicted value and the label of the predicted value; the expression of the second loss function is:

；

wherein, Is the predicted value, Y is a label of the predicted value,/>Is the integrated representation, lambda, gamma,/>Delta is the second weight, the third weight, the fourth weight and the fifth weight, respectively.

5. The neural network-based internet of things asset identification method according to claim 4, wherein predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation comprises:

Obtaining at least one of the integrated representations;

Inputting at least one of the integrated representations into the initial asset identification model for forward propagation, processing through a plurality of intermediate layers until reaching an output layer;

And obtaining a first characteristic output by the output layer, and processing the first characteristic through an activation function to obtain the predicted value.

6. The neural network-based internet of things asset identification method of claim 5, wherein the expression of the predicted value is:

；

wherein, For the predicted value,/>For the integrated representation, the output of the fully connected layer, i.e., the first feature,/>, via the initial asset identification modelFor the weight of the output layer,/>Is the bias of the output layer.

7. The neural network-based internet of things asset identification method according to any one of claims 1-6, wherein the identifying and acquiring image data of an internet of things device in an internet of things network comprises:

Acquiring an original image of the Internet of things equipment through image acquisition equipment;

Acquiring a GAN generator, and inputting the original image to the GAN generator;

And processing the original image through the GAN generator to obtain a simulation image similar to the original image, and taking the original image and the simulation image as the image data.

8. The neural network-based internet of things asset identification method of claim 1, further comprising, prior to said predicting the integrated representation by the initial asset identification model:

Clustering a plurality of integrated representations by adopting a clustering algorithm to obtain a clustering label corresponding to each integrated representation; the cluster label is used for representing a cluster corresponding to the integrated representation;

Partitioning at least one of the integrated representations having the same cluster tag into the same panel;

and ordering the plurality of integrated representations in the order of the groups so that the initial asset identification model preferentially processes the integrated representations belonging to the same group.

9. An internet of things asset identification system based on a neural network, the system comprising:

The data identification module is used for identifying and acquiring image data and flow data of the Internet of things equipment in the Internet of things network;

the feature extraction module is used for carrying out encoding and decoding processing on the image data and the flow data through an automatic encoder of the initial asset identification model to obtain image features and flow features;

the feature fusion module is used for carrying out fusion processing on the image features and the flow features to generate a fused integrated representation;

The model prediction module is used for predicting the integrated representation through the initial asset identification model to obtain a predicted value corresponding to the integrated representation;

The loss construction module is used for constructing a loss function of the initial asset identification model according to the integrated representation, the predicted value and the label of the predicted value;

The model training module is used for training the initial asset identification model according to a training set containing a plurality of image data and a plurality of flow data, the labels of the training set and the loss function until a trained asset identification model is obtained;

The asset identification module is used for inputting an image to be identified and flow to be identified of the equipment of the internet of things to be identified into the asset identification model to obtain the asset category of the equipment of the internet of things to be identified.

10. The neural network-based internet of things asset identification system of claim 9, wherein the feature extraction module is further configured to:

invoking an encoding unit and a decoding unit of the automatic encoder;

Mapping the image data and the flow data to a potential space through the encoding unit to obtain potential characteristics of the image data and the flow data;

And mapping the potential features back to the original data space of the image data and the flow data through the decoding unit so as to reconstruct the image data and the flow data and obtain the corresponding image features and flow features.