CN115599873A - Data acquisition method and system based on artificial intelligence Internet of things and cloud platform - Google Patents
Data acquisition method and system based on artificial intelligence Internet of things and cloud platform Download PDFInfo
- Publication number
- CN115599873A CN115599873A CN202211553105.0A CN202211553105A CN115599873A CN 115599873 A CN115599873 A CN 115599873A CN 202211553105 A CN202211553105 A CN 202211553105A CN 115599873 A CN115599873 A CN 115599873A
- Authority
- CN
- China
- Prior art keywords
- data
- network
- internet
- things
- data cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
According to the data acquisition method, system and cloud platform based on the artificial intelligence Internet of things, the relation network description vectors and the data cluster network node description vectors are used for obtaining the data significance factors of each first data cluster network node, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relation network, the representative data cluster network nodes can be accurately determined in the process of selecting each first data cluster network node based on the reference data significance factors, the Internet of things data packets are analyzed and refined based on the obtained data cluster network node description vectors and the relation network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification flow is effectively improved.
Description
Technical Field
The application relates to the technical field of artificial intelligence and the Internet of things, in particular to a data acquisition method and system based on the artificial intelligence Internet of things and a cloud platform.
Background
With the wider application of artificial intelligence and the internet of things, the internet of things equipment generates a large amount of data, and the data are analyzed and tracked by artificial intelligence and machine learning. In this way, artificial intelligence is combined with the internet of things to create intelligent devices and make informed decisions without human intervention. The possibilities offered by the internet of things are unlimited, and the rapid expansion of networked devices and sensors makes the amount of data they create exponentially growing, and the problem that how to analyze these massive performance data is obviously unrealistic by human power, while machine learning in artificial intelligence is an effective solution. Before analyzing the data, the data acquisition is a precondition, and in the data acquisition process, the data needs to be classified and stored so as to be conveniently and subsequently extracted in a targeted manner according to the analysis requirement.
Disclosure of Invention
The invention aims to provide a data acquisition method, a data acquisition system and a cloud platform based on an artificial intelligence Internet of things so as to improve the efficiency of data classification and storage.
The embodiment of the application achieves the aim as follows:
in a first aspect, an embodiment of the present application provides a data acquisition method based on an artificial intelligence internet of things, which is characterized in that the data acquisition method is applied to a data acquisition cloud platform, the data acquisition cloud platform is in communication connection with at least one internet of things sensor network, the internet of things sensor network includes an internet of things relational database, the internet of things relational database is used for storing internet of things data packages, and the method includes:
when an internet of things data packet uploaded by the internet of things sensor network is acquired, analyzing the internet of things data packet to acquire a first internet of things data relation network, wherein the first internet of things data relation network comprises a plurality of first data cluster network nodes;
acquiring a first relation network description vector of the first internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first internet of things data relation network; the first data cluster network nodes are network nodes corresponding to data cluster information in the data packet of the Internet of things, and the first Internet of things data relation network is established based on the involvement conditions among the plurality of first data cluster network nodes;
processing the first relational network description vector and the plurality of first data cluster network node description vectors to determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network; wherein the data significance factor characterizes representative information of the first data cluster network node in the first Internet of things data relationship network;
determining a plurality of second data clustering nodes in the plurality of first data clustering nodes based on the obtained plurality of data saliency factors, wherein the data saliency factors corresponding to the plurality of second data clustering nodes are larger than the data saliency factors corresponding to the rest of the first data clustering nodes;
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes and a second relation network description vector of a second relation data relation network to determine storage classification information of the data packet of the internet of things; wherein the second associative data relationship network is established based on the involvement between the plurality of second data cluster network nodes.
As a possible implementation manner, after the step of determining a plurality of second data cluster nodes in the plurality of first data cluster nodes based on the obtained plurality of data saliency factors, the method further includes:
modifying a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relational network description vector to obtain a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors to determine a data significance factor of each second data cluster network node in the second relational data network;
determining a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the obtained plurality of data significance factors, wherein the data significance factors corresponding to the plurality of third data cluster network nodes are larger than the data significance factors corresponding to the rest of second data cluster network nodes.
As a possible implementation manner, the extracting a first data cluster node description vector of the plurality of second data cluster nodes and a second data relationship network description vector of a second data relationship network to determine storage classification information of the internet of things data packet includes:
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement between the plurality of third data cluster network nodes.
As a possible implementation manner, the extracting the first data cluster node description vector of the second data cluster nodes, the second relation network description vector, the second data cluster node description vector of the third data cluster nodes, and the third relation network description vector of the third internet of things node to determine the storage classification information of the internet of things packet includes:
based on the numerical statistical result of the second data cluster network nodes, performing mean calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes to determine data cluster network node mean description vectors corresponding to the plurality of second data cluster network nodes;
connecting the data cluster network node mean description vector with a maximum data cluster network node description vector in a first data cluster network node description vector of the plurality of second data cluster network nodes to obtain a first combined description vector;
splicing second data clustering network node description vectors of the plurality of third data clustering network nodes and the third relation network description vector to obtain a second combined description vector;
connecting the first merging description vector with the second merging description vector, and determining a merging description vector corresponding to the Internet of things data packet;
and extracting the merged description vector, and determining storage classification information of the Internet of things data packet.
As a possible implementation manner, the step of processing the first relational network description vector and the plurality of first data cluster network description vectors by the data saliency determination module, and determining the data saliency corresponding to each first data cluster network junction in the first internet of things data relational network is performed by a storage classification network debugged in advance, where the storage classification network includes a first data saliency determination module, a first network junction selection module, and a classification module, and the step of processing the first relational network description vector and the plurality of first data cluster network junction description vectors, and determining the data saliency corresponding to each first data cluster network junction in the first internet of things data relational network specifically includes:
processing the first relational network description vector and the plurality of first data cluster network node description vectors through the first data significance factor determining module, and determining a data significance factor corresponding to each first data cluster network node in the first internet of things data relational network;
the determining a plurality of second data cluster nodes among the plurality of first data cluster nodes based on the obtained plurality of data saliency factors includes:
determining, by the first network node selection module, the plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the obtained plurality of data saliency factors;
the extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second relation data relation network to determine the storage classification information of the internet of things data packet includes:
and extracting the first data clustering network node description vector and the second relation network description vector of the plurality of second data clustering network nodes through the classification module, and determining the storage classification information of the data packet of the internet of things.
The storage and classification network further includes a first linear transformation module, a second data saliency factor determination module, and a second network node selection module, and after the first network node selection module determines the plurality of second data clustering network nodes among the plurality of first data clustering network nodes based on the obtained plurality of data saliency factors, the method further includes:
modifying, by the first linear transformation module, a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relationship network description vector, and determining a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors through the second data significant factor determination module to determine a data significant factor of each second data cluster network node in the second relational network;
determining, by the second network node selection module, a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the obtained plurality of data significance factors, where the data significance factors corresponding to the plurality of third data cluster network nodes are greater than the data significance factors corresponding to the remaining second data cluster network nodes.
As a possible implementation manner, the extracting, by the classification module, the first data cluster node description vector and the second relationship network description vector of the plurality of second data cluster nodes to determine storage classification information of the internet of things data packet includes:
extracting, by the classification module, a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes, and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement conditions among the plurality of third data cluster network nodes.
As a possible implementation manner, the storage classification network further includes a first splicing module and a second splicing module, and the determining, by the classification module, storage classification information of the internet of things data packet by extracting a first data clustering network node description vector of the second data clustering network nodes, the second relation network description vector, a second data clustering network node description vector of the third data clustering network nodes, and a third relation network description vector of a third internet of things data relation network includes:
splicing a first data cluster network node description vector and a second relation network description vector of the plurality of second data cluster network nodes through the first splicing module to obtain a first combined description vector;
splicing a second data cluster network node description vector and the third relation network description vector of the plurality of third data cluster network nodes through the second splicing module to obtain a second combined description vector;
extracting the first merging description vector and the second merging description vector through the classification module, and determining storage classification information of the data packet of the Internet of things;
the storage classification network further comprises a merging module, and the extracting of the first merging description vector and the second merging description vector is performed through the classification module to determine the storage classification information of the data packet of the internet of things, wherein the merging module comprises:
connecting the first merging description vector and the second merging description vector through the merging module, and determining a merging description vector corresponding to the data packet of the internet of things;
and extracting the merged description vector through the classification module, and determining the storage classification information of the data packet of the Internet of things.
As a possible implementation manner, the storage classification network is obtained by debugging the following steps:
acquiring training storage classification information of an Internet of things training data packet, a training relation network description vector of a training Internet of things data relation network and a training data cluster network node description vector of a plurality of training data cluster network nodes in the training Internet of things data relation network, wherein the training data cluster network node is a network node corresponding to the training data cluster information, and the training Internet of things data relation network is established based on the involvement condition among the plurality of training data cluster information in the Internet of things training data packet;
extracting the training relation network description vector and the training data clustering network node description vectors of the training data clustering network nodes through the storage classification network, and determining inference storage classification information of the training data packet of the Internet of things;
and debugging the network parameters of the storage classification network until reaching a preset condition based on the loss between the training storage classification information and the reasoning storage classification information to obtain the storage classification network after debugging.
In a second aspect, an embodiment of the present application provides a data acquisition system, including a data processing cloud platform and at least one internet of things sensor network in communication connection with the data processing cloud platform, the internet of things sensor network includes an internet of things relational database, the internet of things relational database is used for storing internet of things data packets, the data processing cloud platform includes a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, the method described above is performed.
In a third aspect, an embodiment of the present application provides a data acquisition cloud platform, which includes a processor and a memory, where the memory stores a computer program, and when the processor executes the computer program, the method described above is performed.
According to the data acquisition method, the system and the cloud platform based on the artificial intelligence Internet of things, the relation network description vectors and the data cluster network node description vectors are used for obtaining the data significance factors of each first data cluster network node, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relation network, the representative data cluster network nodes can be accurately determined in the process of selecting each first data cluster network node based on the reference data significance factors, the data packet of the Internet of things can be analyzed and refined based on the obtained data cluster network node description vectors and the relation network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification flow is effectively improved.
In the description that follows, additional features will be set forth, in part, in the description. These features will be in part apparent to those skilled in the art upon examination of the following and the accompanying drawings, or may be learned by production or use. The features of the present application may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities and combinations particularly pointed out in the detailed examples which follow.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a flowchart of a data acquisition method based on an artificial intelligence internet of things according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a functional module architecture of a data acquisition device according to an embodiment of the present application.
Fig. 3 is a schematic composition diagram of a data acquisition cloud platform provided in an embodiment of the present application.
Detailed Description
The embodiments of the present application will be described below with reference to the drawings. The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments herein only and is not intended to be limiting of the application.
In the embodiment of the application, an execution subject of the data acquisition method based on the artificial intelligence internet of things is a data acquisition cloud platform, including but not limited to a network server, a server group consisting of a plurality of network servers, or a cloud consisting of a large number of computers or network servers in cloud computing, wherein the cloud computing is one of distributed computing and is a super virtual computer consisting of a group of loosely coupled computers. The data acquisition cloud platform can operate independently to achieve the application, and can also be accessed to a network and achieve the application through interactive operation with other data acquisition cloud platforms in the network. The network where the data acquisition cloud platform is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network and the like; the data acquisition cloud platform is in communication connection with at least one Internet of things sensing network, the Internet of things sensing network comprises an Internet of things relation database, the Internet of things relation database is used for storing Internet of things data packets, the Internet of things sensing network senses and arranges the Internet of things data packets according to a fixed communication protocol through sensors distributed in a target area, the arranged Internet of things data packets are stored in the Internet of things relation database and then sent to the data acquisition cloud platform, and the Internet of things data packets sent by the Internet of things sensing network can be sent in real time or non-real time, such as regular (sent according to a preset period) or quantitative (sent when the data volume reaches a preset size) sending mode. The data in the data packet of the internet of things may be static data or dynamic data, and may be energy type data (related to energy consumption or related data required for calculating energy consumption), asset attribute type data (hardware asset data), diagnostic type data (data for detecting an operation state of the device during operation), and signal type data. The data acquisition cloud platform and the Internet of things sensing network jointly form a data acquisition system.
The embodiment of the application provides a data acquisition method based on an artificial intelligence internet of things, which is applied to a data acquisition cloud platform and comprises the following steps:
step S100: when the Internet of things data packet uploaded by the Internet of things sensor network is obtained, the Internet of things data packet is analyzed to obtain a first Internet of things data relation network, and the first Internet of things data relation network comprises a plurality of first data cluster network nodes.
In the embodiment of the application, the internet of things data packets uploaded by the internet of things sensor network are data packets needing to be classified and stored, namely, the type of the attribution of the internet of things data packets needs to be determined and then stored in the storage space of the same type of data, so that subsequent data processing is facilitated, and the data acquisition process is completed. The data packet of the internet of things comprises a plurality of data clusters, for example, the data of the internet of things contained in each data cluster is the data sensed by the corresponding sensor, and the data packet of the internet of things is formed by each data cluster. In the internet of things data packets formed by the data clusters, the classification contributions of different data clusters to the whole internet of things data packet are different, for example, the data volume proportion, the preset data requirement satisfaction condition and the like.
Step S200: the method comprises the steps of obtaining a first relation network description vector of a first Internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first Internet of things data relation network.
The first internet of things data relation network is established based on the involvement between a plurality of first data cluster network nodes, for example, an internet of things data packet comprises data clusters 1 to 10, wherein the data cluster 1 contains type A sensor data, the data cluster 2 contains type B sensor data, the data cluster 3 contains time sequence data, the data cluster 4 contains coordinate data 8230, the data cluster 8230, wherein the time stamps of the data in the data cluster 1 and the data cluster 2 are recorded in the data cluster 3, the coordinate information is recorded in the data cluster 4, the data cluster 1 and the data cluster 3 and the data cluster 4 are involved (can generate correlation), the data cluster 2 and the data cluster 3 and the data cluster 4 are involved, and the data cluster 1 and the data cluster 2 can be involved or not involved. If the two data clusters involve, the two data clusters are connected, in other words, the first data network nodes corresponding to the data clusters are connected, and finally the first Internet of things data relation network is obtained. The first data cluster network node is a network node corresponding to data cluster information in the data packet of the internet of things, and the data cluster information is information indicating why the data cluster is a data cluster, and may be, for example, a tag or a numerical value. As an implementation manner, if the plurality of first data cluster nodes are data cluster nodes corresponding to all data cluster information in the data packet of the internet of things, in other words, the plurality of first data cluster nodes are not selected, the first internet of things data relationship network is an original internet of things data relationship network, and if the plurality of first data cluster nodes are data cluster nodes corresponding to part of data cluster information in the data packet of the internet of things, in other words, the plurality of first data cluster nodes are already selected, the first internet of things data relationship network is a selected internet of things data relationship network. In the embodiment of the present application, a first relational network description vector (a vector expression embodying features) is used to describe a first internet of things data relational network, and a first data cluster network node description vector is used to describe a data cluster corresponding to data cluster information, for example, the first data cluster network node description vector includes information content of the data cluster and attributes of data of the data cluster.
Step S300: and processing the first relation network description vector and the plurality of first data cluster network node description vectors to obtain a data significance factor corresponding to each first data cluster network node in the first Internet of things data relation network.
The data significance factor represents representative information of the first data cluster network node in the first internet of things data relationship network, in other words, it indicates representative information of data cluster information corresponding to the first data cluster network node in an internet of things data packet, or importance, the data significance factor can be expressed in a numerical manner, the representative information of the data cluster information in the internet of things data packet is positively correlated with the data significance factor, that is, the larger the data significance factor is, the stronger the representative information is, for example, a plurality of sensors arranged in the same bit field, the most important core sensor is for analysis purposes, the representative of the generated data cluster is the highest, and the greater the significance factor corresponding to the data cluster is.
Step S400: and determining a plurality of second data clustering nodes from the plurality of first data clustering nodes based on the obtained plurality of data saliency factors.
The data significance factor corresponding to the plurality of second data clustering nodes is greater than the data significance factor corresponding to the rest of the first data clustering nodes. In step 400, a plurality of first data network nodes are selected, a plurality of corresponding second data network nodes with a larger data significance factor (for example, reaching a preset value) are determined, and then processing is performed based on the selected second data network nodes with a larger data significance factor, for example, information with poor representativeness is cleaned, so that data to be processed is reduced, and representative data is highlighted.
Step S500: and extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second relation data relation network to obtain the storage classification information of the data packet of the Internet of things.
The second data relation network is established based on the involvement between the plurality of second data cluster network nodes, and it can be correspondingly understood that the second relation network description vector can describe the features of the second data relation network, the second data relation network is a part of the first data relation network, the involvement between the plurality of second data cluster network nodes and the plurality of second data cluster network nodes covered in the second data relation network, and the connection between the plurality of second data cluster network nodes in the second data relation network is consistent with the connection between the plurality of second data cluster network nodes in the first data relation network.
The storage classification information represents classification information corresponding to a storage space in which the data packet of the internet of things needs to be stored, and the obtained storage classification information is different through preset classification elements and classification conditions. For example, the preset classification elements are divided according to regions, so that the acquisition regions can be classified according to region information corresponding to the data cluster information in the data packet of the internet of things, and the preset classification elements are divided according to a numerical range, so that the acquisition numerical values are classified according to a numerical interval corresponding to the data cluster information in the data packet of the internet of things.
In the steps S100 to S500, the data significant factor of each first data cluster network node is obtained through the relationship network description vector and the data cluster network node description vector, richer reference information is covered, and the obtained data significant factor can fully reflect the representativeness of the data cluster network node in the internet of things data relationship network, so that a representative data cluster network node can be accurately determined in the subsequent process of selecting each first data cluster network node based on the reference data significant factor, and the internet of things data packet is analyzed and refined based on the obtained data cluster network node description vector and the relationship network description vector of the representative data cluster network node, so that the accuracy of data classification is increased, the data cluster network node with poor representativeness is not included in the category analysis, and the efficiency of the whole storage classification process is effectively improved.
As an implementation manner, the data processing cloud platform stores and classifies the internet of things data packets through a storage and classification network, the storage and classification network may be any feasible artificial intelligence network, such as a deep learning neural network, and specifically may include a data loading module, a first data significant factor determining module, a first net knot selecting module and a classification module, wherein the data loading module is connected with the first data significant factor determining module, the first data significant factor determining module is connected with the first net knot selecting module, and the first net knot selecting module is connected with the classification module. The data loading module is configured to obtain input relational network description vectors and data clustering network node description vectors, the first data saliency factor determination module is configured to obtain a data saliency factor of each data clustering network node, the first network node selection module is configured to determine a representative data clustering network node based on the obtained data saliency factor, and the classification module is configured to determine storage classification information based on the determined data clustering network node description vectors of the data clustering network nodes and the corresponding relational network description vectors.
As an implementation manner, the storage and classification network further includes a first linear transformation module, a second data significant factor determination module, a second network node selection module and a merging module, wherein the first linear transformation module is connected to the first network node selection module and the second data significant factor determination module, the second data significant factor determination module is connected to the second network node selection module, the second network node selection module is connected to the classification module, and the merging module is connected to the first network node selection module, the second network node selection module and the classification module. The first linear transformation module is configured to perform deep processing on data cluster network node description vectors of representative data cluster network nodes determined by the first network node selection module, the deep processing of the linear transformation module can be convolution operation, the second data significance factor determination module is configured to obtain a data significance factor of each determined data cluster network node, the second network node selection module is configured to perform further screening on the determined data cluster network nodes according to the obtained data significance factors, and the combination module is configured to connect the data cluster network node description vectors of the data cluster network nodes determined by the first network node selection module and the data cluster network node description vectors of the data cluster network nodes determined again by the second network node selection module. It can be reasoned that in other embodiments, more filtering picks can be made on a data cluster to get more richness of information. The number of the corresponding data saliency determination modules and network node selection modules becomes larger, and the data cluster network node description vectors of the data cluster network nodes determined by more network node selection modules are connected through the merging module.
As an implementation manner, a linear transformation module may be arranged before the first data significant factor determination module, and configured to perform convolution operation on the input data cluster network structure description vector and the relationship network description vector, where the first data significant factor determination module determines the data significant factor according to the convolved data cluster network structure description vector and the relationship network description vector.
Based on the network architecture, the data acquisition method based on the artificial intelligence internet of things provided by the embodiment of the application can comprise the following steps:
step S10: and establishing a first Internet of things data relation network according to the involvement condition among a plurality of data cluster information in the Internet of things data packet.
Taking an example that a first internet of things data relationship network is an internet of things data relationship network established first as an illustration, in the embodiment of the present application, corresponding data cluster network nodes are established based on a plurality of data cluster information included in an internet of things data packet, and then, the data cluster network nodes are connected based on involvement between the plurality of data cluster information, so as to weave a first internet of things data relationship network, where the first internet of things data relationship network includes only a single type of network node of the data cluster network node, and in addition, the data cluster network nodes are connected based on the same type of involvement, in other words, the connection relationship in the first internet of things data relationship network is a consistent connection relationship, where the involvement between two random data cluster network nodes depends on an involvement degree vector between the two data cluster network nodes, and the involvement degree vector represents a degree of correlation between data clusters corresponding to the two data cluster network nodes, for example, the degree of correlation between the data cluster 1 and the data cluster 2 is low, and the degree of correlation between the data cluster 1 and the data cluster 3 is high.
S20: the method comprises the steps of obtaining a first relation network description vector of a first Internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first Internet of things data relation network.
The first relational network description vector represents a plurality of data cluster network nodes in a first internet of things data relational network and the involvement condition between the data cluster network nodes, the first relational network description vector comprises a correlation degree vector between two random data cluster network nodes in the data cluster network nodes, the first data cluster network node description vector comprises an information content description vector of a data cluster and an attribute description vector of data cluster data, the information content description vector of the data cluster represents the data expression meaning of the data cluster, and the attribute description vector of the data cluster data represents the data attribute information of the data cluster, such as data format and data size, and the information content description vector and the attribute description vector of the data cluster data can extract feature vectors in a universal feature extraction mode.
As an implementation manner, a plurality of first data cluster network node description vectors may be connected to form a matrix, where the content in the vertical and horizontal relationships represents the first data cluster network node description vectors of the first data cluster network nodes, and similarly, the first relationship network description vectors may also form a matrix, where each value represents the involvement of two data cluster network nodes.
S30: and processing the first relation network description vector and the plurality of first data cluster network node description vectors through a first data significant factor determining module to obtain a data significant factor corresponding to each first data cluster network node in the first Internet of things data relation network.
The first data significant factor determining module may be a module formed by a neural network of a graph, and the process of obtaining the data significant factor may refer to a general local output function (local output function).
S40: and determining a plurality of second data cluster network nodes from the plurality of first data cluster network nodes based on the obtained plurality of data significant factors through a first network node selection module, and establishing a second associated data relation network based on the involvement between the plurality of second data cluster network nodes.
The data significance factor corresponding to a plurality of second data clustering network nodes is larger than the data significance factor corresponding to the rest first data clustering network nodes. As an implementation manner, a first network node selection module multiplies the numerical statistical results of a plurality of first data cluster network nodes by a preset parameter to obtain reference numerical statistical results, arranges the data significant factors of the plurality of first data cluster network nodes according to the numerical values, determines the data significant factors of the plurality of reference numerical statistical results with larger numerical values, determines the first data cluster network nodes corresponding to the determined data significant factors as a plurality of obtained second data cluster network nodes, and establishes a second associated data relationship network based on the obtained involvement between the plurality of second data cluster network nodes after the plurality of second data cluster network nodes are obtained.
Based on consideration of the data significance factor, the first data cluster network node description vectors of the plurality of second data cluster network nodes may be modified to obtain modified first data cluster network node description vectors, for example, the first data cluster network node description vectors of the second data cluster network nodes are multiplied by the corresponding data significance factor to obtain modified first data cluster network node description vectors.
The above process is explained based on multiple selections of the data cluster nodes, and if the data cluster nodes are selected only once, the storage classification information of the data packet of the internet of things can be obtained directly based on the data cluster node description vectors of the second data cluster nodes and the second relation network description vectors for establishing the second relation data relation network after the second data cluster nodes are determined.
S50: and modifying the first data cluster network node description vectors of the plurality of second data cluster network nodes through the first linear transformation module based on the second relation network description vectors to obtain the second data cluster network node description vectors of the plurality of second data cluster network nodes.
The second data cluster network junction description vector may be obtained by multiplying several matrices, i.e. multiplying the second relation network description vector, the first data cluster network junction description vector of the second data cluster network junction and the parameters of the first linear transformation module.
S60: and processing the second relational network description vector and the plurality of second data cluster network node description vectors through a second data significant factor determining module to obtain a data significant factor of each second data cluster network node in the second relational network.
S70: and determining a plurality of third data cluster network nodes from the plurality of second data cluster network nodes through a second network node selection module based on the obtained plurality of data significant factors, and establishing a third internet of things data relationship network according to the involvement conditions among the plurality of third data cluster network nodes.
S80: and extracting the first data cluster network node description vectors, the second relation network description vectors, the second data cluster network node description vectors and the third relation network description vectors of the third data cluster network nodes of the plurality of second data cluster network nodes through the classification module to obtain the storage classification information of the data packet of the Internet of things.
As an implementation manner, based on consideration of pressure of processing data by the reduction classification module, a first data cluster network node description vector and a second relation network description vector of a plurality of second data cluster network nodes may be spliced to obtain a first merged description vector, and a second data cluster network node description vector and a third relation network description vector of a plurality of third data cluster network nodes may be spliced to obtain a second merged description vector; and extracting the first merging description vector and the second merging description vector through a classification module to obtain storage classification information of the data packet of the Internet of things.
As an implementation manner, based on the numerical statistical result of the second data cluster network nodes, performing mean calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes to obtain data cluster network node mean description vectors corresponding to the plurality of second data cluster network nodes, and connecting the data cluster network mean description vectors with the largest data cluster network node description vector in the first data cluster network node description vectors of the plurality of second data cluster network nodes to obtain first combined description vectors. And for a second data cluster network node description vector and a third relation network description vector of a third data cluster network node, performing mean value calculation on a second data cluster network node description vector and a third relation network description vector of a plurality of third data cluster network nodes according to a numerical value statistical result of the third data cluster network node to obtain a data cluster network node mean value description vector corresponding to the plurality of third data cluster network nodes, and connecting the data cluster network node mean value description vector with a maximum data cluster network node description vector in the second data cluster network node description vectors of the plurality of third data cluster network nodes to obtain a second combined description vector.
As an implementation manner, the first merged description vector and the second merged description vector are connected to obtain a merged description vector corresponding to the internet of things data packet, and the merged description vector is extracted to obtain storage classification information corresponding to the internet of things data packet. The merged description vector is the feature representing the data packet of the internet of things.
As an implementation mode, the storage classification network further comprises a first splicing module, a second splicing module and a merging module, wherein the first splicing module splices a first data clustering network node description vector and a second relation network description vector of a plurality of second data clustering network nodes to obtain a first merging description vector, the second splicing module splices a second data clustering network node description vector and a third relation network description vector of a plurality of third data clustering network nodes to obtain a second merging description vector, and the merging module connects the first merging description vector and the second merging description vector to obtain a merging description vector corresponding to the data packet of the internet of things; and extracting the combined description vector through a classification module to obtain storage classification information of the data packet of the Internet of things.
According to the data acquisition method based on the artificial intelligence Internet of things, the data significance factors of the first data cluster network nodes are obtained through the relation network description vectors and the data cluster network node description vectors, richer reference information is covered, the representativeness of the data cluster network nodes in the Internet of things data relation network can be fully reflected by the obtained data significance factors, the representative data cluster network nodes can be accurately determined in the process of selecting the first data cluster network nodes based on the reference data significance factors, the Internet of things data packets are analyzed and refined based on the obtained data cluster network node description vectors and the relation network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification process is effectively improved. In addition, multiple selections are performed on the data clustering network nodes, data clustering network node description vectors and relation network description vectors of different levels are obtained, and when the data packet of the Internet of things is analyzed, the data clustering network node description vectors and the relation network description vectors of different levels are considered, so that the classification accuracy is further improved.
As an implementation manner, the embodiment of the present application further provides a debugging process of the storage classification network.
The debugging process of the storage classification network comprises the following steps: acquiring training storage classification information of an Internet of things training data packet, training relation network description vectors of a training Internet of things data relation network and training data cluster network node description vectors of a plurality of training data cluster network nodes in the training Internet of things data relation network; extracting the training relation network description vector and the training data cluster network node description vectors of the training data cluster network nodes through a storage classification network to obtain inference storage classification information of the training data packet of the Internet of things, and debugging network parameters of the storage classification network until a preset condition is met based on loss between the training storage classification information and the inference storage classification information to obtain the storage classification network after debugging. The training data cluster network node is a network node corresponding to training data cluster information, the training internet of things data relation network is established based on the involvement condition among a plurality of pieces of training data cluster information in the internet of things training data packet, and the preset condition can be that the debugging turn reaches preset times or the inference precision of the network reaches preset precision.
In order to improve the inference precision of the network, after the storage classification information of the data packet of the internet of things is analyzed through the storage classification network, the storage classification network can be further debugged according to the data packet of the internet of things.
As an implementation manner, after the internet of things data packet is stored and classified, a process of verifying integrity and correctness of the internet of things data, that is, a process of cleaning the internet of things data, may be further included, where the process includes the following steps:
step S600: the method comprises the steps of obtaining knowledge fields of a plurality of data sets of the internet of things to be cleaned, and determining the knowledge field of each data set of the internet of things to be cleaned as a connecting terminal.
The data sets of the internet of things to be cleaned can be a set formed by classified data of the internet of things, a plurality of data sets of the internet of things to be cleaned can be loaded into a convolutional neural network debugged in advance as a whole data set of the internet of things to be cleaned, and a knowledge field of each data set of the internet of things to be cleaned is acquired and used for representing characteristic information of the data sets of the internet of things.
Step S700: each connection terminal is connected to an adjacent connection terminal.
The neighboring connection terminals of the connection terminals may be obtained based on a commonality measurement result (representing a degree of similarity between the connection terminals), for example, calculating a vector distance between the connection terminals, the closer the distance, the higher the commonality measurement result. For example, computing the cosine distance between vectors. As an embodiment, for each connection terminal, M connection terminals with the largest result of the measure of commonality with the connection terminal are obtained as the adjacent connection terminals of the connection terminal, M is selected depending on the amount of data to be classified, each connection terminal is connected to the corresponding adjacent connection terminal, and then a plurality of connection lines are obtained, one of which connects two connection terminals.
Step S800: and acquiring the probability of the same classification information of the connecting terminal and the adjacent connecting terminal.
In the embodiment of the present application, the probability refers to the possibility that a connection terminal belongs to a target classification information, and it is evaluated whether the connection terminal and the similar region are sufficiently similar, and the similar region represents the region generated by the adjacent connection terminal of each connection terminal, and is the same classification information. For example, a connection terminal with a high probability determines that there are other connection terminals and that they are the same classification information, in other words, a connection terminal with a high probability has a high probability of being a classification information for itself, whereas a connection terminal with a low probability cannot determine whether there is a connection terminal and that it is the same classification information, in other words, a connection terminal with a low probability has a low probability of being a target classification information for itself.
As an embodiment, the probability that the connection terminal and the adjacent connection terminal are the same classification information is a difference between a first result of the commonality measurement that is a sum of results of the commonality measurements that the adjacent connection terminal and the connection terminal of the connection terminal are the same classification information and a second result of the commonality measurement that the adjacent connection terminal and the connection terminal of the connection terminal are a sum of results of the commonality measurements that are different classification information. For connection terminals for which classification information cannot be determined, a probabilistic inference network can be implemented to infer the probability of the connection terminal based on the completion of the debugging.
Step S900: the degree of association of the connection line between the connection terminal and each adjacent connection terminal is determined based on the likelihood of the two connection terminals being connected being the same classification information.
And if the connecting lines with strong association degree reflect that the possibility that the two connected connecting terminals are the same classification information is high, the association degree of the connecting lines with the possibility that the two connected connecting terminals are the same classification information being 1 is set as 1, otherwise, the association degree of the connecting lines with the possibility that the two connected connecting terminals are the same classification information being 0 is set as 0, the association degree of the connecting lines between the two connecting terminals which cannot determine the classification information is 0 to 1, and the association degree of the connecting lines is deduced through an association reasoning network which is debugged in advance.
Step S1000: and based on the probability of each connecting terminal and the association degree of each connecting wire, classifying the plurality of connecting terminals which are mutually mapped with the plurality of Internet of things data sets to be cleaned to obtain classification clusters of the plurality of Internet of things data sets to be cleaned.
For the adjacent data sets for determining the probability of the connecting terminals and the association degree of the connecting lines, a plurality of networking data sets to be cleaned corresponding to the connecting terminals can be classified (namely clustered) through a preset classification algorithm.
In one embodiment, a connection line between a connection terminal and a target adjacent connection terminal may be obtained, the adjacent connection terminal including the target adjacent connection terminal, and further, a probability of the target adjacent connection terminal is greater than that of the connection terminal and the connection line between the target adjacent connection terminal and the connection terminal includes a maximum degree of association. In other words, for each connection terminal, the connection line obtained is generated between the adjacent connection terminal having a probability greater than that of the connection terminal and the connection terminal, and the connection line includes the greatest degree of association among the connection lines generated by all the adjacent connection terminals and the connection terminals. Classifying the data set of the to-be-cleaned internet of things corresponding to the connecting terminal based on a through interval generated by the connecting line between the connecting terminal and the target adjacent connecting terminal to obtain at least one connecting terminal classification set, in other words, at least one cluster. Each through interval represents classification information, or each connection terminal classification set corresponds to the classification information, and classification clusters of a plurality of Internet of things data sets to be cleaned can be obtained based on at least one connection terminal classification set.
Step S1100: and cleaning the data of the Internet of things which are dissociated outside each classification cluster.
And removing the Internet of things data outside each classification cluster, namely the isolated points among the classification clusters, namely the Internet of things data noise, and then finishing the cleaning of the data.
In the embodiment of the application, the knowledge fields of a plurality of Internet of things data sets to be cleaned are used as the connecting terminals, each connecting terminal is connected with the adjacent connecting terminals to obtain a plurality of connecting wires, based on the probability of determining the connecting terminals and the association degree of the connecting wires, a plurality of Internet of things data sets to be cleaned corresponding to the connecting terminals are classified, the classifying process is changed into a process for reasoning the probability of the connecting terminals and the association degree of the connecting wires, and the classifying efficiency and accuracy are improved.
As an implementation mode, the probability of each connecting terminal is inferred through a preset probabilistic inference network, a knowledge field matrix corresponding to a plurality of connecting terminals is obtained on the basis of knowledge fields corresponding to the connecting terminals, then a commonality measurement result matrix is obtained on the basis of a commonality measurement result between every two connecting terminals, and then the knowledge field matrix and the commonality measurement result matrix are loaded to the probabilistic inference network debugged in advance to obtain the probability that the connecting terminals and adjacent connecting terminals are the same classification information, wherein the probabilistic inference network can be a graph neural network.
As an embodiment, the probabilistic inference network may be debugged through an internet of things training data set, where each internet of things training data subset in the internet of things training data set is labeled with classification information (i.e., type).
For the training data set of the internet of things, the probability corresponding to each training data subset of the internet of things is obtained, the contents can be referred to, for the training data set of the internet of things, the knowledge field of each training data subset of the internet of things is obtained, the knowledge field of each training data subset of the internet of things is used as a training connecting terminal, for each training connecting terminal, the probability that the training connecting terminal and the adjacent connecting terminal are the same classification information is determined, and the probability is marked for the training connecting terminal. And then, acquiring training knowledge field matrixes corresponding to a plurality of training connection terminals based on the knowledge fields corresponding to the training connection terminals.
And obtaining a training common measurement result matrix based on the common measurement result between every two training connection terminals, loading the training knowledge field matrix and the training common measurement result matrix into a probabilistic inference network, and inferring to obtain the probability of each training connection terminal. And adjusting the network parameters of the probabilistic inference network based on the loss between the probability obtained by reasoning each training connection terminal and the labeled probability until a preset condition is met, such as model convergence or debugging times.
As an embodiment, the association degree of the connecting line between each connecting terminal and each adjacent connecting terminal is inferred through an association inference network debugged in advance. For each connection terminal, a set of alternatives is determined, the set of alternatives comprising neighboring connection terminals of the connection terminal, the probability being greater than the neighboring connection terminals of the connection terminal. The probability of the adjacent connection terminal in the close region is greater than that of the connection terminal, which represents that the adjacent connection terminal is more likely to classify information for one object, in order to classify the connection terminal into the feature classification information, the connection terminal may be connected to the adjacent connection terminal having a higher probability than that of the connection terminal, an alternative set is determined for the connection terminal, but the selected adjacent connection terminal and the connection terminal may not be the same classification information, and then consideration of the degree of association of the connection terminal needs to be introduced. And then, loading the alternative set to a correlation inference network debugged in advance, wherein the correlation inference network outputs the degree of correlation between the connecting terminal and each connecting line between the nearest adjacent connecting terminals in the alternative set.
The associative inference network and the probabilistic inference network may have similar architectures, but the probabilistic inference network does not process the entire adjacent data set, the associative inference network processes the sub-adjacent data set constructed by the candidate set, and the associative inference network generates a probability that each adjacent connection terminal in the candidate set represents that the adjacent connection terminal and the connection terminal are the same classification information.
As an implementation manner, the associative inference network may be debugged through an internet of things training data set, where each internet of things training data subset in the internet of things training data set is labeled with classification information. The method comprises the steps of obtaining a knowledge field of each training data subset of the Internet of things, using the knowledge field of each training data subset of the Internet of things as a training connecting terminal, and obtaining a training alternative set for each training connecting terminal, wherein the alternative set comprises adjacent connecting terminals of the training connecting terminals, and the probability of the alternative set is larger than that of the adjacent connecting terminals of the training connecting terminals. Then connecting the training connecting terminal with each adjacent connecting terminal in the alternative set to obtain a training connecting line, determining the association degree of the training connecting line based on the classification information of the training connecting terminal and the classification information of the adjacent connecting terminal, marking the training connecting line, determining the association degree of the connecting line between the training connecting terminal and the adjacent connecting terminal as 1 when the classification information of the training connecting terminal and the adjacent connecting terminal is consistent, and determining the association degree between the training connecting terminal and the adjacent connecting terminal as 0 when the classification information of the training connecting terminal and the adjacent connecting terminal is inconsistent; and loading the alternative set to an associated reasoning network, reasoning to obtain the association degree of each connecting line, and adjusting the network parameters of the associated reasoning network until reaching a preset condition based on the loss between the association degree and the labeled association degree of each training connecting line to obtain the debugged associated reasoning network.
Based on the same principle as the method shown in fig. 1, the embodiment of the present application further provides a data acquisition apparatus 10, as shown in fig. 2, where the apparatus 10 includes:
the analysis module 11 is configured to, when acquiring the internet of things data packet uploaded by the internet of things sensor network, analyze the internet of things data packet to obtain a first internet of things data relationship network, where the first internet of things data relationship network includes a plurality of first data cluster network nodes.
The mining module 12 is configured to obtain a first relationship network description vector of the first internet of things data relationship network and a first data cluster network node description vector of a plurality of first data cluster network nodes in the first internet of things data relationship network. The first data cluster network nodes are network nodes corresponding to data cluster information in the data packet of the Internet of things, and the first Internet of things data relation network is established based on the involvement conditions among the plurality of first data cluster network nodes.
A data significant factor determining module 13, configured to process the first relational network description vector and the plurality of first data cluster network node description vectors, and determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network. Wherein the data significance factor characterizes representative information of the first data cluster network node in the first Internet of things data relationship network.
The screening module 14 is configured to determine, based on the obtained multiple data significance factors, multiple second data cluster network nodes from among the multiple first data cluster network nodes, where data significance factors corresponding to the multiple second data cluster network nodes are greater than data significance factors corresponding to the remaining first data cluster network nodes.
And the storage classification module 15 is used for extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second relation data relation network to determine the storage classification information of the internet of things data packet. Wherein the second associative data relationship network is established based on the involvement between the plurality of second data cluster network nodes.
The foregoing embodiment introduces the data acquisition apparatus 10 from the perspective of a virtual module, and the following introduces a data acquisition cloud platform from the perspective of a physical module, as follows:
an embodiment of the present application provides a data acquisition cloud platform, as shown in fig. 3, the data acquisition cloud platform 100 includes: a processor 101 and a memory 103. Wherein the processor 101 is coupled to the memory 103, such as via a bus 102. Optionally, the data acquisition cloud platform 100 may further include a transceiver 104. It should be noted that, in practical applications, the transceiver 104 is not limited to one, and the structure of the data acquisition cloud platform 100 does not constitute a limitation to the embodiments of the present application.
The processor 101 may be a CPU, general purpose processor, GPU, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein. The processor 101 may also be a combination of computing functions, e.g., comprising one or more microprocessors in combination, a DSP and a microprocessor in combination, or the like.
The memory 103 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The memory 103 is used for storing application program codes for executing the scheme of the application, and the execution is controlled by the processor 101. The processor 101 is configured to execute application program code stored in the memory 103 to implement the aspects of any of the method embodiments described above.
The embodiment of the application provides a data acquisition cloud platform, and the data acquisition cloud platform in the embodiment of the application comprises: one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more programs are executed by the processors, the method for data collection based on artificial intelligence internet of things as described above is performed. According to the technical scheme, the data significance factors of the first data cluster network nodes are obtained through the relational network description vectors and the data cluster network node description vectors, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relational network, the representative data cluster network nodes can be accurately determined in the process of selecting the first data cluster network nodes based on the reference data significance factors, the Internet of things data packets are analyzed and refined based on the obtained data cluster network node description vectors and the relational network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification process is effectively improved.
The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a processor, enables the processor to execute the corresponding content in the foregoing method embodiments.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (10)
1. The data acquisition method based on the artificial intelligence Internet of things is applied to the data acquisition cloud platform, the data acquisition cloud platform is in communication connection with at least one Internet of things sensing network, the Internet of things sensing network comprises an Internet of things relation database, the Internet of things relation database is used for storing Internet of things data packets, and the method comprises the following steps:
when an internet of things data packet uploaded by the internet of things sensor network is acquired, analyzing the internet of things data packet to acquire a first internet of things data relation network, wherein the first internet of things data relation network comprises a plurality of first data cluster network nodes;
acquiring a first relation network description vector of the first internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first internet of things data relation network; the first data cluster network node is a network node corresponding to data cluster information in the data packet of the internet of things, and the first data relation network of the internet of things is established based on the involvement condition among the plurality of first data cluster network nodes;
processing the first relational network description vector and the plurality of first data cluster network node description vectors to determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network; wherein the data significance factor characterizes representative information of the first data cluster network node in the first Internet of things data relationship network;
determining a plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the obtained plurality of data significance factors, wherein the data significance factors corresponding to the plurality of second data cluster network nodes are larger than the data significance factors corresponding to the rest of first data cluster network nodes;
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes and a second relation network description vector of a second relation data relation network to determine storage classification information of the data packet of the internet of things; wherein the second associative data relationship network is established based on the involvement between the plurality of second data cluster network nodes.
2. The method of claim 1, wherein after the step of determining a plurality of second data cluster nodes among the plurality of first data cluster nodes based on the obtained plurality of data saliency factors, the method further comprises:
modifying a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relational network description vector to obtain a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors, and determining a data significance factor of each second data cluster network node in the second relational network;
determining a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the obtained plurality of data significant factors, wherein the data significant factors corresponding to the plurality of third data cluster network nodes are larger than the data significant factors corresponding to the rest of the second data cluster network nodes.
3. The method according to claim 2, wherein the extracting the first data cluster node description vector of the plurality of second data cluster nodes and the second relational network description vector of the second relational network to determine the storage classification information of the internet of things data packet comprises:
extracting a first data clustering network node description vector of the second data clustering network nodes, the second relation network description vector, a second data clustering network node description vector of the third data clustering network nodes and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement conditions among the third data clustering network nodes.
4. The method according to claim 3, wherein the extracting the first data cluster network node description vector of the second data cluster network nodes, the second relation network description vector, the second data cluster network node description vector of the third data cluster network nodes, and the third relation network description vector of the third internet of things node to determine the storage classification information of the internet of things data packet comprises:
based on the numerical statistical result of the second data cluster network nodes, performing mean calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes to determine data cluster network node mean description vectors corresponding to the plurality of second data cluster network nodes;
connecting the data cluster network node mean description vector with a maximum data cluster network node description vector in a first data cluster network node description vector of the plurality of second data cluster network nodes to obtain the first combined description vector;
splicing second data clustering network node description vectors of the plurality of third data clustering network nodes and the third relation network description vector to obtain a second combined description vector;
connecting the first merging description vector with the second merging description vector, and determining a merging description vector corresponding to the Internet of things data packet;
and extracting the merged description vector, and determining storage classification information of the Internet of things data packet.
5. The method according to claim 1, wherein the data saliency determination module network node selection module classifies the first relational network description vector and the plurality of first data cluster network node description vectors, and the step of determining the data saliency corresponding to each first data cluster network node in the first internet of things data relational network is performed by a pre-debugged storage classification network, which includes a first data saliency determination module, a first network node selection module, and a classification module, and the step of processing the first relational network description vector and the plurality of first data cluster network node description vectors to determine the data saliency corresponding to each first data cluster network node in the first internet of things data relational network specifically includes:
processing the first relational network description vector and the plurality of first data cluster network node description vectors through the first data significant factor determination module, and determining a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network;
the determining a plurality of second data cluster nodes among the plurality of first data cluster nodes based on the obtained plurality of data saliency factors includes:
determining, by the first network node selection module, the plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the obtained plurality of data saliency factors;
the extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vector of the second relation data relation network to determine the storage classification information of the internet of things data packet includes:
and extracting the first data clustering network node description vector and the second relation network description vector of the plurality of second data clustering network nodes through the classification module, and determining the storage classification information of the data packet of the internet of things.
The storage and classification network further includes a first linear transformation module, a second data saliency factor determination module, and a second network node selection module, and after the first network node selection module determines the plurality of second data clustering network nodes among the plurality of first data clustering network nodes based on the obtained plurality of data saliency factors, the method further includes:
modifying, by the first linear transformation module, a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relationship network description vector, and determining a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors through the second data significance factor determining module, and determining a data significance factor of each second data cluster network node in the second relational network;
determining, by the second network node selection module, a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the plurality of acquired data significance factors, where data significance factors corresponding to the plurality of third data cluster network nodes are greater than data significance factors corresponding to the remaining second data cluster network nodes.
6. The method according to claim 5, wherein the extracting, by the classification module, the first data cluster node description vector and the second relationship network description vector of the plurality of second data cluster nodes to determine the storage classification information of the internet of things data packet comprises:
extracting, by the classification module, a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes, and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement conditions among the plurality of third data cluster network nodes.
7. The method according to any one of claims 5 to 6, wherein the storage classification network further comprises a first splicing module and a second splicing module, and the determining the storage classification information of the IOT data packet by extracting a first data clustering net node description vector of the plurality of second data clustering net nodes, the second relation net description vector, a second data clustering net node description vector of the plurality of third data clustering net nodes and a third relation net description vector of a third IOT data relation net through the classification module comprises:
splicing a first data cluster network node description vector and a second relation network description vector of the plurality of second data cluster network nodes through the first splicing module to obtain a first combined description vector;
splicing a second data cluster network node description vector and the third relation network description vector of the plurality of third data cluster network nodes through the second splicing module to obtain a second combined description vector;
extracting the first merging description vector and the second merging description vector through the classification module, and determining storage classification information of the data packet of the Internet of things;
the storage classification network further includes a merging module, and the extracting the first merged description vector and the second merged description vector through the classification module to determine the storage classification information of the internet of things data packet includes:
connecting the first merging description vector and the second merging description vector through the merging module, and determining a merging description vector corresponding to the data packet of the internet of things;
and extracting the merged description vector through the classification module, and determining the storage classification information of the data packet of the Internet of things.
8. The method according to any one of claims 5 to 7, wherein the storage classification network is obtained by debugging the following steps:
acquiring training storage classification information of an Internet of things training data packet, training relation network description vectors of a training Internet of things data relation network and training data cluster network node description vectors of a plurality of training data cluster network nodes in the training Internet of things data relation network, wherein the training data cluster network nodes are network nodes corresponding to the training data cluster information, and the training Internet of things data relation network is established based on the involvement condition among the plurality of training data cluster information in the Internet of things training data packet;
extracting the training relation network description vector and the training data clustering network node description vectors of the training data clustering network nodes through the storage classification network, and determining inference storage classification information of the training data packet of the Internet of things;
and debugging the network parameters of the storage classification network until a preset condition is reached based on the loss between the training storage classification information and the reasoning storage classification information to obtain the storage classification network after debugging.
9. A data acquisition system is characterized by comprising a data processing cloud platform and at least one Internet of things sensing network in communication connection with the data processing cloud platform, wherein the Internet of things sensing network comprises an Internet of things relation database which is used for storing Internet of things data packets, the data processing cloud platform comprises a processor and a memory, a computer program is stored in the memory, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is executed.
10. A data acquisition cloud platform comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, performs the method of any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211553105.0A CN115599873B (en) | 2022-12-06 | 2022-12-06 | Data acquisition method and system based on artificial intelligence Internet of things and cloud platform |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211553105.0A CN115599873B (en) | 2022-12-06 | 2022-12-06 | Data acquisition method and system based on artificial intelligence Internet of things and cloud platform |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115599873A true CN115599873A (en) | 2023-01-13 |
CN115599873B CN115599873B (en) | 2023-08-29 |
Family
ID=84851927
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211553105.0A Active CN115599873B (en) | 2022-12-06 | 2022-12-06 | Data acquisition method and system based on artificial intelligence Internet of things and cloud platform |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115599873B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117273765A (en) * | 2023-11-21 | 2023-12-22 | 广州欧派创意家居设计有限公司 | Multistage dealer circulation data processing method and system based on automatic check |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9311386B1 (en) * | 2013-04-03 | 2016-04-12 | Narus, Inc. | Categorizing network resources and extracting user interests from network activity |
WO2018219223A1 (en) * | 2017-05-31 | 2018-12-06 | 腾讯科技(深圳)有限公司 | Data processing method, device and storage medium |
CN109033387A (en) * | 2018-07-26 | 2018-12-18 | 广州大学 | A kind of Internet of Things search system, method and storage medium merging multi-source data |
CN109347834A (en) * | 2018-10-24 | 2019-02-15 | 广东工业大学 | Detection method, device and the equipment of abnormal data in Internet of Things edge calculations environment |
CN109672750A (en) * | 2019-01-15 | 2019-04-23 | 特斯联(北京)科技有限公司 | A kind of security protection data aggregate management system and its management method based on Edge fabrics |
CN110348526A (en) * | 2019-07-15 | 2019-10-18 | 武汉绿色网络信息服务有限责任公司 | A kind of device type recognition methods and device based on semi-supervised clustering algorithm |
CN110955212A (en) * | 2019-12-05 | 2020-04-03 | 河南工业大学 | Wisdom agricultural information processing system based on thing networking |
CN112104684A (en) * | 2020-02-12 | 2020-12-18 | 云池网络科技(上海)有限公司 | Platform management system based on Internet of things |
CN113207101A (en) * | 2021-04-13 | 2021-08-03 | 山东曙光照信息技术有限公司 | Information processing method based on 5G city component sensor and Internet of things cloud platform |
US20210352134A1 (en) * | 2020-01-19 | 2021-11-11 | Bernt Erik Bjontegard | Contextual intelligence of every-thing communication platform integrating edge computation, augmented intelligence and distributed ledger data storage nodes and tokens for real world data interactions across internet of things with payment integration |
CN113837009A (en) * | 2021-08-26 | 2021-12-24 | 张大艳 | Internet of things data acquisition and analysis system based on artificial intelligence |
CN114257980A (en) * | 2021-12-06 | 2022-03-29 | 安徽南瑞继远电网技术有限公司 | Data interaction system and interaction method based on cluster network |
CN114547440A (en) * | 2022-01-25 | 2022-05-27 | 王金山 | User portrait mining method based on internet big data and artificial intelligence cloud system |
-
2022
- 2022-12-06 CN CN202211553105.0A patent/CN115599873B/en active Active
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9311386B1 (en) * | 2013-04-03 | 2016-04-12 | Narus, Inc. | Categorizing network resources and extracting user interests from network activity |
WO2018219223A1 (en) * | 2017-05-31 | 2018-12-06 | 腾讯科技(深圳)有限公司 | Data processing method, device and storage medium |
CN109033387A (en) * | 2018-07-26 | 2018-12-18 | 广州大学 | A kind of Internet of Things search system, method and storage medium merging multi-source data |
CN109347834A (en) * | 2018-10-24 | 2019-02-15 | 广东工业大学 | Detection method, device and the equipment of abnormal data in Internet of Things edge calculations environment |
CN109672750A (en) * | 2019-01-15 | 2019-04-23 | 特斯联(北京)科技有限公司 | A kind of security protection data aggregate management system and its management method based on Edge fabrics |
CN110348526A (en) * | 2019-07-15 | 2019-10-18 | 武汉绿色网络信息服务有限责任公司 | A kind of device type recognition methods and device based on semi-supervised clustering algorithm |
CN110955212A (en) * | 2019-12-05 | 2020-04-03 | 河南工业大学 | Wisdom agricultural information processing system based on thing networking |
US20210352134A1 (en) * | 2020-01-19 | 2021-11-11 | Bernt Erik Bjontegard | Contextual intelligence of every-thing communication platform integrating edge computation, augmented intelligence and distributed ledger data storage nodes and tokens for real world data interactions across internet of things with payment integration |
CN112104684A (en) * | 2020-02-12 | 2020-12-18 | 云池网络科技(上海)有限公司 | Platform management system based on Internet of things |
CN113207101A (en) * | 2021-04-13 | 2021-08-03 | 山东曙光照信息技术有限公司 | Information processing method based on 5G city component sensor and Internet of things cloud platform |
CN113837009A (en) * | 2021-08-26 | 2021-12-24 | 张大艳 | Internet of things data acquisition and analysis system based on artificial intelligence |
CN114257980A (en) * | 2021-12-06 | 2022-03-29 | 安徽南瑞继远电网技术有限公司 | Data interaction system and interaction method based on cluster network |
CN114547440A (en) * | 2022-01-25 | 2022-05-27 | 王金山 | User portrait mining method based on internet big data and artificial intelligence cloud system |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117273765A (en) * | 2023-11-21 | 2023-12-22 | 广州欧派创意家居设计有限公司 | Multistage dealer circulation data processing method and system based on automatic check |
CN117273765B (en) * | 2023-11-21 | 2024-02-06 | 广州欧派创意家居设计有限公司 | Multistage dealer circulation data processing method and system based on automatic check |
Also Published As
Publication number | Publication date |
---|---|
CN115599873B (en) | 2023-08-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109978228B (en) | PM2.5 concentration prediction method, device and medium | |
CN108615071B (en) | Model testing method and device | |
CN112396160A (en) | Transaction fraud detection method and system based on graph neural network | |
CN113378961A (en) | Network traffic identification method, device, equipment and computer program product | |
CN110809768B (en) | Data cleansing system and method | |
CN116596095B (en) | Training method and device of carbon emission prediction model based on machine learning | |
CN115599873A (en) | Data acquisition method and system based on artificial intelligence Internet of things and cloud platform | |
CN111292377B (en) | Target detection method, device, computer equipment and storage medium | |
CN115062779A (en) | Event prediction method and device based on dynamic knowledge graph | |
CN112257332B (en) | Simulation model evaluation method and device | |
CN117131348B (en) | Data quality analysis method and system based on differential convolution characteristics | |
WO2024078112A1 (en) | Method for intelligent recognition of ship outfitting items, and computer device | |
CN114565092A (en) | Neural network structure determining method and device | |
CN112580780A (en) | Model training processing method, device, equipment and storage medium | |
CN109508582A (en) | The recognition methods of remote sensing image and device | |
CN115484112B (en) | Payment big data safety protection method, system and cloud platform | |
CN114860617B (en) | Intelligent pressure testing method and system | |
CN115984671A (en) | Model online updating method and device, electronic equipment and readable storage medium | |
CN112528500A (en) | Evaluation method and evaluation equipment for scene graph construction model | |
CN112819527A (en) | User grouping processing method and device | |
CN115712614B (en) | Information processing method and system based on data transmission flow control | |
CN112580781A (en) | Processing method, device and equipment of deep learning model and storage medium | |
TWI613545B (en) | Analyzing method and analyzing system for graphics process | |
CN113537407B (en) | Image data evaluation processing method and device based on machine learning | |
CN110750661B (en) | Method, device, computer equipment and storage medium for searching image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20230807 Address after: 510000 room 6-211, Part 5, 6 and 7, No. 67, Dongpu Second Road, Tianhe District, Guangzhou City, Guangdong Province (office only) Applicant after: Guangzhou Fengwang Internet Technology Co.,Ltd. Address before: No. 109, Ankang Street, Bincheng District, Binzhou City, Shandong Province, 256600 - 0085 Applicant before: Binzhou Yuwei Network Technology Co.,Ltd. |
|
GR01 | Patent grant | ||
GR01 | Patent grant |