CN115599873B - Data acquisition method and system based on artificial intelligence Internet of things and cloud platform - Google Patents

Data acquisition method and system based on artificial intelligence Internet of things and cloud platform Download PDF

Info

Publication number
CN115599873B
CN115599873B CN202211553105.0A CN202211553105A CN115599873B CN 115599873 B CN115599873 B CN 115599873B CN 202211553105 A CN202211553105 A CN 202211553105A CN 115599873 B CN115599873 B CN 115599873B
Authority
CN
China
Prior art keywords
data
network
data cluster
internet
things
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211553105.0A
Other languages
Chinese (zh)
Other versions
CN115599873A (en
Inventor
陈城
陈劲峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Fengwang Internet Technology Co ltd
Original Assignee
Guangzhou Fengwang Internet Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Fengwang Internet Technology Co ltd filed Critical Guangzhou Fengwang Internet Technology Co ltd
Priority to CN202211553105.0A priority Critical patent/CN115599873B/en
Publication of CN115599873A publication Critical patent/CN115599873A/en
Application granted granted Critical
Publication of CN115599873B publication Critical patent/CN115599873B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

According to the data acquisition method, system and cloud platform based on the artificial intelligence Internet of things, the data salient factors of all the first data cluster nets are acquired through the relational network description vector and the data cluster net description vector, richer reference information is covered, the acquired data salient factors can fully represent the representativeness of the data cluster nets in the Internet of things data relational network, the representative data cluster nets can be accurately determined in the subsequent process of selecting all the first data cluster nets based on the reference data salient factors, the data package of the Internet of things is analyzed based on the acquired data cluster net description vector and the relational network description vector of the representative data cluster net, accuracy of data classification is improved, the representative data cluster net with poor representativeness is not included in an analysis category, and efficiency of the whole storage classification flow is effectively improved.

Description

Data acquisition method and system based on artificial intelligence Internet of things and cloud platform
Technical Field
The application relates to the technical field of artificial intelligence and the Internet of things, in particular to a data acquisition method, a system and a cloud platform based on the artificial intelligence Internet of things.
Background
As the use of artificial intelligence and internet of things has increased, internet of things devices have generated a vast amount of data that is analyzed and tracked by artificial intelligence and machine learning. In this way, artificial intelligence is combined with the internet of things, creating intelligent devices, and making intelligent decisions without human intervention. The possibilities offered by the internet of things are unlimited, the rapid expansion of networking devices and sensors, such that the amount of data they create will grow exponentially, with the consequent problem of how to analyze these massive performance data, which is obviously impractical by human power, while machine learning in artificial intelligence is an effective solution. Before analyzing the data, the data is collected on the premise that the data needs to be classified and stored in the data collection process so that the data can be extracted in a targeted mode according to analysis requirements, and the efficiency of classified storage of the data of the Internet of things is a problem that attention is required.
Disclosure of Invention
The invention aims to provide a data acquisition method, a system and a cloud platform based on an artificial intelligence Internet of things so as to improve the efficiency of data classification and storage.
The embodiment of the application achieves the aim according to the following content:
in a first aspect, an embodiment of the present application provides a data collection method based on an artificial intelligence internet of things, which is characterized in that the method is applied to a data collection cloud platform, the data collection cloud platform is in communication connection with at least one internet of things sensing network, the internet of things sensing network includes an internet of things relational database, and the internet of things relational database is used for storing internet of things data packets, the method includes:
when an Internet of things data packet uploaded by the Internet of things sensing network is obtained, analyzing the Internet of things data packet to obtain a first Internet of things data relationship network, wherein the first Internet of things data relationship network comprises a plurality of first data cluster nodes;
acquiring a first relation network description vector of the first Internet of things data relation network and first data cluster network description vectors of a plurality of first data clusters in the first Internet of things data relation network; the first data cluster network node is a network node corresponding to data cluster information in the internet of things data packet, and the first internet of things data relation network is established based on the involvement among the plurality of first data cluster network nodes;
processing the first relation network description vector and the plurality of first data cluster network description vectors, and determining a data significance factor corresponding to each first data cluster network in the first Internet of things data relation network; wherein the data significance factor characterizes representative information of the first data cluster network in the first Internet of things data relationship network;
Determining a plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the plurality of acquired data significant factors, wherein the data significant factors corresponding to the plurality of second data cluster network nodes are larger than the data significant factors corresponding to the rest of first data cluster network nodes;
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes and a second relationship network description vector of a second internet-of-things data relationship network, and determining storage classification information of the internet-of-things data packets; wherein the second linked data relationship network is established based on involvement of the plurality of second data cluster nodes.
As a possible implementation manner, after the step of determining a plurality of second data cluster nodes in the plurality of first data cluster nodes based on the acquired plurality of data saliency factors, the method further includes:
correcting the first data cluster network node description vectors of the plurality of second data cluster network nodes based on the second relation network description vector to obtain second data cluster network node description vectors of the plurality of second data cluster network nodes;
processing the second relation network description vector and the plurality of second data cluster network node description vectors, and determining a data significance factor of each second data cluster network node in the second internet-of-things data relation network;
And determining a plurality of third data cluster network nodes in the second data cluster network nodes based on the acquired plurality of data significant factors, wherein the data significant factors corresponding to the third data cluster network nodes are larger than the data significant factors corresponding to the rest second data cluster network nodes.
As a possible implementation manner, the extracting the first data cluster network node description vector of the plurality of second data cluster network nodes and the second relationship network description vector of the second connection data relationship network to determine storage classification information of the internet of things data packet includes:
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network node description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes and a third relation network node description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on the involvement among the plurality of third data cluster network nodes.
As a possible implementation manner, the extracting the first data cluster network node description vector of the plurality of second data cluster network nodes, the second relationship network description vector, the second data cluster network node description vector of the plurality of third data cluster network nodes, and the third relationship network description vector of the third internet of things data relationship network, and determining storage classification information of the internet of things data packet includes:
Based on the numerical statistics result of the second data cluster network nodes, carrying out average value calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes, and determining data cluster network node average value description vectors corresponding to the plurality of second data cluster network nodes;
connecting the data cluster network node mean value description vector with the largest data cluster network node description vector in the first data cluster network node description vectors of the plurality of second data cluster network nodes to obtain the first merging description vector;
splicing the second data cluster network node description vectors of the plurality of third data cluster network nodes and the third relation network description vector to obtain a second merging description vector;
connecting the first merging description vector with the second merging description vector, and determining the merging description vector corresponding to the data packet of the Internet of things;
and extracting the combined description vector, and determining storage classification information of the internet of things data packet.
As a possible implementation manner, the step of determining the data saliency factor corresponding to each first data cluster in the first internet of things data relationship network by processing the first relationship network description vector and the plurality of first data cluster network description vectors by the data saliency factor determining module and the classifying module by the classifying module is performed by a pre-debugged storage classifying network, the storage classifying network includes a first data saliency factor determining module, a first network node selecting module and a classifying module, and the step of processing the first relationship network description vector and the plurality of first data cluster network description vectors to determine the data saliency factor corresponding to each first data cluster network node in the first internet of things data relationship network specifically includes:
Processing the first relation network description vector and the plurality of first data cluster network description vectors through the first data significant factor determining module, and determining the data significant factor corresponding to each first data cluster network in the first Internet of things data relation network;
the determining, based on the acquired plurality of data saliency factors, a plurality of second data cluster nodes among the plurality of first data cluster nodes includes:
determining, by the first cluster selection module, the plurality of second data cluster clusters from the plurality of first data cluster clusters based on the plurality of acquired data saliency factors;
extracting the first data cluster network node description vector of the plurality of second data cluster network nodes and the second relationship network description vector of the second link data relationship network, and determining storage classification information of the internet of things data packet, wherein the method comprises the following steps:
and extracting the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes through the classifying module, and determining storage classifying information of the data packets of the Internet of things.
The storage classification network further comprises a first linear transformation module, a second data significant factor determination module and a second network node selection module, wherein the method further comprises, after determining the plurality of second data cluster nodes in the plurality of first data cluster nodes based on the plurality of acquired data significant factors by the first network node selection module:
Correcting the first data cluster network node description vectors of the plurality of second data cluster network nodes based on the second relation network description vectors through the first linear transformation module, and determining second data cluster network node description vectors of the plurality of second data cluster network nodes;
processing the second relation network description vector and the plurality of second data cluster network description vectors through the second data significant factor determining module, and determining the data significant factor of each second data cluster network in the second internet-of-things data relation network;
and determining a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the acquired plurality of data significant factors through the second network node selection module, wherein the data significant factors corresponding to the plurality of third data cluster network nodes are larger than the data significant factors corresponding to the rest of second data cluster network nodes.
As a possible implementation manner, the extracting, by the classifying module, the first data cluster network node description vector and the second relationship network node description vector of the plurality of second data cluster network nodes, and determining storage classification information of the internet of things data packet includes:
And extracting a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network node description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes and a third relation network node description vector of a third Internet of things data relation network by the classifying module, and determining storage classifying information of the Internet of things data packet, wherein the third Internet of things data relation network is established based on the involving conditions among the plurality of third data cluster network nodes.
As a possible implementation manner, the storage classification network further includes a first splicing module and a second splicing module, and the determining, by the classification module, storage classification information of the internet of things data packet includes:
splicing the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors through the first splicing module to obtain a first merging description vector;
Splicing the second data cluster network node description vectors of the plurality of third data cluster network nodes and the third relation network description vectors through the second splicing module to obtain second merging description vectors;
extracting the first combined description vector and the second combined description vector through the classifying module, and determining storage classifying information of the internet of things data packet;
the storage classification network further includes a merging module, and the extracting, by the classification module, the first merging description vector and the second merging description vector, to determine storage classification information of the internet of things data packet includes:
the merging module is used for connecting the first merging description vector and the second merging description vector and determining the merging description vector corresponding to the data packet of the Internet of things;
and extracting the combined description vector through the classifying module, and determining the storage classifying information of the data packet of the Internet of things.
As a possible implementation manner, the storage classification network is obtained by debugging through the following steps:
acquiring training storage classification information of an Internet of things training data packet, training relation network description vectors of a training Internet of things data relation network and training data cluster network description vectors of a plurality of training data clusters in the training Internet of things data relation network, wherein the training data cluster network is a network corresponding to the training data cluster information, and the training Internet of things data relation network is established based on the involvement of the plurality of training data cluster information in the Internet of things training data packet;
Extracting the training relation network description vector and the training data cluster network description vectors of the plurality of training data clusters through the storage classification network, and determining the reasoning storage classification information of the training data packets of the Internet of things;
and debugging network parameters of the storage classification network based on the loss between the training storage classification information and the reasoning storage classification information until a preset condition is reached, so as to obtain a debugged storage classification network.
In a second aspect, an embodiment of the present application provides a data acquisition system, including a data processing cloud platform and at least one internet of things sensing network communicatively connected to the data processing cloud platform, where the internet of things sensing network includes an internet of things relational database, where the internet of things relational database is used to store internet of things data packets, and the data processing cloud platform includes a processor and a memory, where the memory stores a computer program, and when the processor executes the computer program, the method is executed.
In a third aspect, an embodiment of the present application provides a data acquisition cloud platform, including a processor and a memory, where the memory stores a computer program, and when the processor executes the computer program, the method is performed.
According to the data acquisition method, system and cloud platform based on the artificial intelligence Internet of things, the data salient factors of all the first data cluster nets are acquired through the relational network description vector and the data cluster net description vector, richer reference information is covered, the acquired data salient factors can fully represent the representativeness of the data cluster nets in the Internet of things data relational network, the representative data cluster nets can be accurately determined in the subsequent process of selecting all the first data cluster nets based on the reference data salient factors, the data package of the Internet of things is analyzed based on the acquired data cluster net description vector and the relational network description vector of the representative data cluster net, accuracy of data classification is improved, the representative data cluster net with poor representativeness is not included in an analysis category, and efficiency of the whole storage classification flow is effectively improved.
In the following description, other features will be partially set forth. Upon review of the ensuing disclosure and the accompanying figures, those skilled in the art will in part discover these features or will be able to ascertain them through production or use thereof. The features of the present application may be implemented and obtained by practicing or using the various aspects of the methods, tools, and combinations that are set forth in the detailed examples described below.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a flowchart of a data acquisition method based on an artificial intelligence internet of things according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a functional module architecture of a data acquisition device according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a composition of a data acquisition cloud platform according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. The terminology used in the description of the embodiments of the application is for the purpose of describing particular embodiments of the application only and is not intended to be limiting of the application.
The implementation main body of the data acquisition method based on the artificial intelligence internet of things in the embodiment of the application is a data acquisition cloud platform, including but not limited to a network server, a server group formed by a plurality of network servers or a cloud formed by a large number of computers or network servers in cloud computing, wherein the cloud computing is one of distributed computing and is a super virtual computer formed by a group of loosely coupled computer sets. The data acquisition cloud platform can independently operate to realize the application, and can also access a network and realize the application through the interaction operation with other data acquisition cloud platforms in the network. The network where the data acquisition cloud platform is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like; the data acquisition cloud platform is in communication connection with at least one Internet of things sensing network, the Internet of things sensing network comprises an Internet of things relational database, the Internet of things relational database is used for storing Internet of things data packets, the Internet of things sensing network senses and sorts the Internet of things data packets according to a fixed communication protocol through sensors distributed in a target area, the sorted Internet of things data packets are stored in the Internet of things relational database and then are sent to the data acquisition cloud platform, and the Internet of things sensing network can send the Internet of things data packets in real time or in a non-real time mode, for example, periodically (sent according to a preset period) or quantitatively (sent when the data quantity reaches a preset size). The data in the data packet of the internet of things can be static data or dynamic data, and can be energy source data (related to energy consumption or related data required by calculating energy consumption), asset attribute data (hardware asset data), diagnosis data (data for detecting the running state of equipment in the running process) and signal data. The data acquisition cloud platform and the Internet of things sensing network jointly form a data acquisition system.
The embodiment of the application provides a data acquisition method based on an artificial intelligence internet of things, which is applied to a data acquisition cloud platform, as shown in fig. 1, and comprises the following steps:
step S100: when the Internet of things data packet uploaded by the Internet of things sensing network is obtained, the Internet of things data packet is analyzed to obtain a first Internet of things data relationship network, and the first Internet of things data relationship network comprises a plurality of first data cluster nodes.
In the embodiment of the application, the internet of things data packet uploaded by the internet of things sensing network is the data packet needing to be classified and stored, namely the type of the internet of things data packet is determined, and then the data packet is stored in a storage space of the same type of data, so that subsequent data processing is facilitated, and the process of data acquisition is completed. The internet of things data packet comprises a plurality of data clusters, for example, internet of things data contained in each data cluster is data sensed by a corresponding sensor, and each data cluster forms the internet of things data packet. In the data packets of the internet of things formed by the data clusters, the classification contributions of different data clusters to the data packets of the whole internet of things are different, for example, the data volume ratio, the preset data requirement meeting condition and the like.
Step S200: and acquiring a first relation network description vector of the first Internet of things data relation network and first data cluster network description vectors of a plurality of first data cluster networks in the first Internet of things data relation network.
The first internet of things data relation network is established based on the involvement situation among the plurality of first data cluster network nodes, for example, an internet of things data packet comprises data clusters 1-10, wherein the data cluster 1 contains sensor data of type a, the data cluster 2 contains sensor data of type B, the data cluster 3 contains time sequence data, the data cluster 4 contains coordinate data … …, wherein the time stamps of the data in the data cluster 1 and the data cluster 2 are recorded in the data cluster 3, the coordinate information is recorded in the data cluster 4, the data cluster 1 and the data cluster 3 and the data cluster 4 are involved (can generate correlation), the data cluster 2 and the data cluster 3 and the data cluster 4 are involved, and the data cluster 1 and the data cluster 2 can be involved or not involved. If the two data clusters are involved, connecting the two data clusters, in other words, connecting the first data network nodes corresponding to the data clusters, and finally obtaining a first Internet of things data relationship network. The first data cluster is a network node corresponding to data cluster information in the data packet of the internet of things, and the data cluster information indicates why the data cluster is the data cluster, for example, the data cluster information can be a label or a numerical value. As an implementation manner, if the plurality of first data cluster nodes are data cluster nodes corresponding to all data cluster information in the data packet of the internet of things, in other words, the plurality of first data cluster nodes are not selected, the first internet of things data relationship network is an original internet of things data relationship network, and if the plurality of first data cluster nodes are data cluster nodes corresponding to part of data cluster information in the data packet of the internet of things, in other words, the plurality of first data cluster nodes are already selected, the first internet of things data relationship network is a selected and completed internet of things data relationship network. In the embodiment of the present application, the first relational network description vector (representing the vector expression of the feature) is used to describe the first internet of things data relational network, and the first data cluster network description vector describes the data cluster corresponding to the data cluster information, for example, the first data cluster network description vector includes the information content of the data cluster and the attribute of the data cluster data.
Step S300: and processing the first relation network description vector and the plurality of first data cluster network description vectors to obtain a data significance factor corresponding to each first data cluster network in the first Internet of things data relation network.
The data saliency factor characterizes the representative information of the first data cluster network in the first internet of things data relation network, in other words, the representative information or importance of the data cluster information corresponding to the first data cluster network in the internet of things data packet is indicated, the data saliency factor can be expressed in a numerical mode, that is, the larger the representative information is, the stronger the representative information is, for example, a plurality of sensors arranged in the same bit domain are, the core sensor is the most important for analysis purposes, the representative of the generated data cluster is the highest, and then the corresponding saliency factor of the data cluster is larger.
Step S400: and determining a plurality of second data cluster nodes from the plurality of first data cluster nodes based on the plurality of acquired data saliency factors.
The data significance factors corresponding to the plurality of second data cluster nodes are larger than the data significance factors corresponding to the rest of first data cluster nodes. In step 400, a plurality of first data cluster nodes are selected, a plurality of second data cluster nodes with larger corresponding data significance factors (for example, reaching a preset value) are determined, and then the second data cluster nodes with larger selected data significance factors are processed, for example, the information of poor representativeness is cleaned, so that the data to be processed is reduced, and the representative data is highlighted.
Step S500: and extracting the first data cluster network node description vector of the plurality of second data cluster network nodes and the second relation network description vector of the second internet-of-things data relation network to obtain storage classification information of the internet-of-things data packets.
The second linked data relationship network is established based on the involvement of the plurality of second data cluster nodes, and it can be correspondingly understood that the second relationship network description vector can describe the characteristics of the second linked data relationship network, the second linked data relationship network is a part of the first linked data relationship network, the second linked data relationship network covers the plurality of second data cluster nodes and the involvement of the plurality of second data cluster nodes, and the connection relationship between the plurality of second data cluster nodes in the second linked data relationship network and the connection relationship between the plurality of second data cluster nodes in the first linked data relationship network are consistent.
The storage classification information indicates classification information corresponding to a storage space in which the internet of things data packet needs to be stored, and the obtained storage classification information is different through preset classification elements and classification conditions. For example, the preset classification elements are divided according to regions, so that the collection regions can be classified according to the region information corresponding to the data cluster information in the data packet of the internet of things, the preset classification elements are divided according to the numerical range, and the collection values are classified according to the numerical intervals corresponding to the data cluster information in the data packet of the internet of things.
The steps S100-S500 are performed, the data saliency factors of the first data cluster network nodes are obtained through the relational network description vector and the data cluster network node description vector, richer reference information is covered, the obtained data saliency factors can fully represent the representativeness of the data cluster network nodes in the Internet of things data relational network, the representative data cluster network nodes can be accurately determined in the subsequent process of selecting the first data cluster network nodes based on the reference data saliency factors, and the data cluster data packets of the Internet of things are analyzed and refined based on the obtained data cluster network node description vector and the relational network description vector of the representative data cluster network nodes, so that the accuracy of data classification is improved, the representative poor data cluster network nodes are not included in the analysis category, and the efficiency of the whole storage classification flow is effectively improved.
As an implementation manner, the data processing cloud platform stores and classifies the internet of things data packet through a storage classification network, and the storage classification network can be any feasible artificial intelligent network, such as a deep learning neural network, and specifically can comprise a data loading module, a first data significant factor determining module, a first network node selecting module and a classification module, wherein the data loading module is connected with the first data significant factor determining module, the first data significant factor determining module is connected with the first network node selecting module, and the first network node selecting module is connected with the classification module. The data loading module is configured to acquire an input relational network description vector and a data cluster network description vector, the first data saliency factor determining module is configured to acquire a data saliency factor of each data cluster network, the first network node selecting module is configured to determine a representative data cluster network based on the acquired data saliency factors, and the classifying module is configured to determine storage classifying information based on the determined data cluster network description vector of the data cluster network and the corresponding relational network description vector.
As an embodiment, the storage classification network further comprises a first linear transformation module, a second data significant factor determination module, a second network node selection module and a merging module, wherein the first linear transformation module is connected with the first network node selection module and the second data significant factor determination module, the second data significant factor determination module is connected with the second network node selection module, the second network node selection module is connected with the classification module, and the merging module is connected with the first network node selection module, the second network node selection module and the classification module. The first linear transformation module is configured to perform deep processing on the data cluster mesh description vector of the representative data cluster mesh determined by the first mesh selection module, the deep processing of the linear transformation module may be a convolution operation, the second data saliency factor determination module is configured to acquire the data saliency factor of each determined data cluster mesh, the second mesh selection module is configured to further screen the determined data cluster mesh according to the acquired data saliency factors, and the merging module is configured to connect the data cluster mesh description vector of the data cluster mesh determined by the first mesh selection module with the data cluster mesh description vector of the data cluster mesh determined again by the second mesh selection module. It can be inferred that in other embodiments, more filtering picks can be made on the data clusters to get enriched information. The number of the corresponding data significant factor determining modules and the network node selecting modules is increased, and the merging module is used for connecting the data cluster network node description vectors of the data cluster network nodes determined by the more network node selecting modules.
As an implementation manner, a linear transformation module may be set before the first data significant factor determining module, and convolution operation is performed on the input data cluster network node description vector and the relationship network description vector, where the first data significant factor determining module determines the data significant factor according to the convolved data cluster network node description vector and the relationship network description vector.
Based on the network architecture, the data acquisition method based on the artificial intelligence internet of things provided by the embodiment of the application can comprise the following steps:
step S10: and establishing a first Internet of things data relation network according to the involvement situation among the plurality of data cluster information in the Internet of things data packet.
Taking the first internet of things data relationship network as an example of the first established internet of things data relationship network, in the embodiment of the application, corresponding data cluster networks are established based on a plurality of data cluster information contained in the internet of things data packet, and then each data cluster network is connected based on the involvement situation among the plurality of data cluster information, so as to weave the first internet of things data relationship network, wherein the first internet of things data relationship network only comprises a single type of network node of the data cluster network, in addition, the data cluster networks are connected based on the same type of involvement situation, in other words, the connection relationship in the first internet of things data relationship network is a consistent connection relationship, wherein the involvement situation between two random data cluster networks depends on the involvement degree vector among the two data cluster networks, the involvement degree vector characterizes the relativity among the data clusters corresponding to the two data cluster networks, the relativity between the data cluster 1 and the data cluster 2 is low, and the relativity between the data cluster 1 and the data cluster 3 is high in the above steps.
S20: and acquiring a first relation network description vector of the first Internet of things data relation network and first data cluster network description vectors of a plurality of first data cluster networks in the first Internet of things data relation network.
The first relational network description vector characterizes a plurality of data cluster network nodes in the first Internet of things data relational network and the involvement among the plurality of data cluster network nodes, the first relational network description vector comprises a correlation vector between two random data cluster network nodes in the plurality of data cluster network nodes, the first data cluster network description vector comprises an information content description vector of a data cluster and an attribute description vector of data cluster data, the information content description vector of the data cluster characterizes the data expression meaning of the data cluster, the attribute description vector of the data cluster characterizes the data attribute information of the data cluster, such as data format, data size, and the information content description vector and the attribute description vector of the data cluster data can adopt a general feature extraction mode to extract feature vectors.
As an implementation manner, a plurality of first data cluster network node description vectors can be connected to form a matrix, wherein the content in the longitudinal and transverse relation represents the first data cluster network node description vector of the first data cluster network node, and similarly, the first relation network description vector can also form the matrix, and each numerical value represents the involvement situation of two data cluster network nodes.
S30: and processing the first relation network description vector and the plurality of first data cluster network description vectors through a first data significant factor determining module to obtain the data significant factor corresponding to each first data cluster network in the first Internet of things data relation network.
The first data significant factor determination module may be a module formed by a graph neural network, and the process of obtaining the data significant factor may refer to a general local output function (local output function).
S40: and determining a plurality of second data cluster networks from the plurality of first data cluster networks based on the acquired plurality of data saliency factors by the first network node selection module, and establishing a second linked data relationship network based on the involvement conditions among the plurality of second data cluster networks.
The data significant factors corresponding to the plurality of second data cluster nodes are larger than the data significant factors corresponding to the rest of the first data cluster nodes. As an implementation manner, through a first network node selection module, multiplying a numerical value statistical result of a plurality of first data cluster nodes by a preset parameter to obtain a reference numerical value statistical result, arranging data significant factors of the plurality of first data cluster nodes according to the numerical value, determining a plurality of reference numerical value statistical result data significant factors with larger numerical values, determining the first data cluster node corresponding to the determined data significant factors as a plurality of obtained second data cluster nodes, and establishing a second internet-of-things data relation network based on the involvement conditions among the obtained plurality of second data cluster nodes after the plurality of second data cluster nodes are obtained.
Based on the consideration of the data significance factors, the first data cluster network junction description vectors of the plurality of second data cluster network junctions can be corrected to obtain corrected first data cluster network junction description vectors, for example, the first data cluster network junction description vectors of the second data cluster network junctions are multiplied by the corresponding data significance factors to obtain corrected first data cluster network junction description vectors.
The above process is described based on the fact that a plurality of selections are performed on the data cluster network nodes, if the data cluster network nodes are selected only once, after a plurality of second data cluster network nodes are determined, storage classification information of the internet of things data packets can be obtained directly based on the data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors for establishing the second internet of things data relation network.
S50: and correcting the first data cluster network junction description vectors of the plurality of second data cluster network junctions based on the second relation network description vectors through the first linear transformation module to obtain second data cluster network junction description vectors of the plurality of second data cluster network junctions.
The second data cluster network description vector may be obtained by multiplying several matrices, namely the second relational network description vector, the first data cluster network description vector of the second data cluster network and the parameters of the first linear transformation module.
S60: and processing the second relation network description vector and the plurality of second data cluster network description vectors through a second data significance factor determining module to obtain the data significance factor of each second data cluster network in the second internet-of-things data relation network.
S70: and determining a plurality of third data cluster networks from the plurality of second data cluster networks based on the acquired plurality of data saliency factors through a second network node selection module, and establishing a third Internet of things data relation network according to the involvement conditions among the plurality of third data cluster networks.
S80: and extracting the first data cluster network junction description vector, the second relation network description vector, the second data cluster network junction description vector and the third relation network description vector of the third data cluster network junction by the classifying module to obtain the storage classifying information of the data packet of the Internet of things.
As an implementation manner, based on the consideration of the pressure of the data processing by the reduction classification module, the first data cluster network node description vectors and the second relationship network description vectors of the plurality of second data cluster network nodes can be spliced to obtain a first merging description vector, and the second data cluster network node description vectors and the third relationship network description vectors of the plurality of third data cluster network nodes can be spliced to obtain a second merging description vector; and extracting the first combined description vector and the second combined description vector through the classifying module to obtain storage classifying information of the data packet of the Internet of things.
As one implementation mode, based on a numerical value statistical result of the second data cluster, carrying out average value calculation on first data cluster network description vectors and second relation network description vectors of the second data cluster network nodes to obtain data cluster network node average value description vectors corresponding to the second data cluster network nodes, and connecting the data cluster network node average value description vectors with the largest data cluster network node description vector in the first data cluster network node description vectors of the second data cluster network nodes to obtain a first merging description vector. And carrying out mean value calculation on the second data cluster mesh description vectors and the third relation mesh description vectors of the third data cluster mesh according to the numerical statistics result of the third data cluster mesh to obtain data cluster mean value description vectors corresponding to the third data cluster mesh, and connecting the data cluster mean value description vectors with the largest data cluster mesh description vector in the second data cluster mesh description vectors of the third data cluster mesh to obtain a second merging description vector.
As an implementation mode, the first merging description vector is connected with the second merging description vector to obtain a merging description vector corresponding to the data packet of the Internet of things, and the merging description vector is extracted to obtain storage classification information corresponding to the data packet of the Internet of things. The combined description vector is the characteristic of the data packet representing the Internet of things.
As an implementation manner, the storage classification network further comprises a first splicing module, a second splicing module and a merging module, the first splicing module is used for splicing the first data cluster network junction description vectors and the second relation network description vectors of the plurality of second data cluster networks to obtain a first merging description vector, the second splicing module is used for splicing the second data cluster network junction description vectors and the third relation network description vectors of the plurality of third data cluster networks to obtain a second merging description vector, and the merging module is used for connecting the first merging description vector and the second merging description vector to obtain a merging description vector corresponding to the data packet of the internet of things; and extracting the combined description vector through a classification module to obtain storage classification information of the data packet of the Internet of things.
According to the data acquisition method based on the artificial intelligence Internet of things, the data salient factors of all the first data cluster networks are acquired through the relational network description vector and the data cluster network description vector, richer reference information is covered, the acquired data salient factors can fully represent the representativeness of the data cluster networks in the Internet of things data relational network, the representative data cluster networks can be accurately determined in the subsequent process of selecting all the first data cluster networks based on the reference data salient factors, the data packet of the Internet of things is analyzed and refined based on the acquired data cluster network description vector and the relational network description vector of the representative data cluster network, the accuracy of data classification is improved, the representative data cluster network with poor representativeness is not included in an analysis category, and the efficiency of the whole storage classification flow is effectively improved. In addition, the data cluster network nodes are selected for multiple times, so that data cluster network node description vectors and relationship network description vectors of different levels are obtained, and when the data packets of the Internet of things are analyzed, the data cluster network node description vectors and the relationship network description vectors of different levels are considered, so that the classification accuracy is further improved.
As an implementation manner, the embodiment of the application further provides a debugging process for storing the categorizing network.
The debugging process of the storage classification network comprises the following steps: acquiring training storage classification information of a training data packet of the Internet of things, training relation network description vectors of a training Internet of things data relation network and training data cluster network description vectors of a plurality of training data clusters in the training Internet of things data relation network; extracting training relation network description vectors and training data cluster description vectors of a plurality of training data cluster nodes through a storage classification network to obtain reasoning storage classification information of the training data packets of the Internet of things, and debugging network parameters of the storage classification network until preset conditions are reached based on losses between the training storage classification information and the reasoning storage classification information to obtain a debugged storage classification network. The training data cluster network is a network node corresponding to training data cluster information, the training internet of things data relation network is established based on the involvement of a plurality of training data cluster information in the internet of things training data packet, and the preset condition can be that the number of times of debugging reaches preset times or the reasoning precision of the network reaches preset precision.
In order to improve the reasoning precision of the network, after the storage classification information of the internet of things data packet is analyzed through the storage classification network, the storage classification network can be further debugged according to the internet of things data packet.
As an implementation manner, after storing and classifying the internet of things data packet, the method may further include a process of verifying the integrity and the correctness of the internet of things data, that is, a process of cleaning the internet of things data, which includes the following steps:
step S600: and acquiring knowledge fields of a plurality of to-be-cleaned Internet of things data sets, and determining the knowledge field of each to-be-cleaned Internet of things data set as a connecting terminal.
The to-be-cleaned internet of things data sets can be sets formed by the classified internet of things data, a plurality of to-be-cleaned internet of things data sets can be used as a whole to-be-cleaned internet of things data sets to be loaded into a convolutional neural network which is debugged in advance, knowledge fields of each to-be-cleaned internet of things data set are obtained, and the knowledge fields are used for representing characteristic information of the internet of things data sets.
Step S700: each connection terminal is connected to an adjacent connection terminal.
Adjacent connection terminals of the connection terminals may be obtained based on a commonality measurement result (representing a degree of similarity of the two) between the connection terminals, for example, by calculating a vector distance between the two, the closer the distance, the higher the commonality measurement result. For example, the cosine distance between vectors is calculated. As an embodiment, for each connection terminal, M connection terminals having the greatest commonality measurement result with the connection terminal are obtained as adjacent connection terminals of the connection terminal, the selection of M depends on the amount of data to be categorized, each connection terminal is connected with the corresponding adjacent connection terminal, and a plurality of connection lines are obtained, wherein one connection line connects two connection terminals.
Step S800: and acquiring the probability that the connection terminal and the adjacent connection terminal are the same classification information.
In the embodiment of the application, the probability refers to the probability that the connection terminal belongs to one target classification information, whether the connection terminal is close enough to a similar region is evaluated, and meanwhile, the similar region represents the region generated by the adjacent connection terminal of each connection terminal for the same classification information. For example, a connection terminal with a large probability determines that there are other connection terminals and it is the same classification information, in other words, a connection terminal with a large probability has a large probability of being one classification information for itself, whereas a connection terminal with a small probability cannot determine whether there are connection terminals and it is the same classification information, in other words, a connection terminal with a small probability has a small probability of being one target classification information for itself.
As one embodiment, the probability that the connection terminal and the adjacent connection terminal are the same classification information is a difference between a first common measurement result, which is a sum of common measurement results of the adjacent connection terminal and the connection terminal of the connection terminal being the same classification information, and a second common measurement result, which is a sum of common measurement results of the adjacent connection terminal and the connection terminal of the connection terminal being different classification information. For connection terminals for which classification information cannot be determined, a network may be inferred based on the probability of achieving debug completion to infer the probability of the connection terminal.
Step S900: the degree of association of the connection terminals with the connection lines between each adjacent connection terminal is determined based on the likelihood that the two connected connection terminals are the same classification information.
The connection line with strong association degree reflects that the possibility that two connection terminals are the same classified information is high, the association degree of the connection line with the possibility that the two connection terminals are the same classified information is set to be 1, otherwise, the association degree of the connection line with the possibility that the two connection terminals are the same classified information is set to be 0, the association degree of the connection line between the two connection terminals which cannot determine the classified information is set to be 0, the association degree of the connection line is inferred through an association inference network which is debugged in advance.
Step S1000: based on the probability of each connecting terminal and the association degree of each connecting wire, classifying the plurality of connecting terminals which are mutually mapped with the plurality of data sets of the internet of things to be cleaned, and obtaining classifying clusters of the plurality of data sets of the internet of things to be cleaned.
For adjacent data sets for determining the probability of the connecting terminals and the association degree of the connecting lines, a plurality of to-be-cleaned Internet of things data sets corresponding to the connecting terminals can be classified (i.e. clustered) through a preset classification algorithm.
As an embodiment, a connection line between a connection terminal and a target adjacent connection terminal may be obtained, the adjacent connection terminal including the target adjacent connection terminal, and further, the probability of the target adjacent connection terminal is greater than the connection terminal and the connection line between the target adjacent connection terminal and the connection terminal includes the maximum degree of association. In other words, for each connection terminal, the connection line obtained is produced between the adjacent connection terminal and the connection terminal with a probability larger than that of the connection terminal, and the connection line includes the maximum degree of correlation among the connection lines produced by all the adjacent connection terminals and the connection terminal. Based on a through section generated by a connecting line between the connecting terminal and the target adjacent connecting terminal, classifying the to-be-cleaned Internet of things data set corresponding to the connecting terminal to obtain at least one connecting terminal classifying set, in other words, at least one cluster. Each through section represents one piece of classifying information, or each connecting terminal classifying set corresponds to one piece of classifying information, and based on at least one connecting terminal classifying set, a plurality of classifying clusters of the to-be-cleaned internet of things data sets can be obtained.
Step S1100: and cleaning the data of the Internet of things which are free outside each classification cluster.
And the data of the Internet of things outside each classification cluster, namely, the isolated points among the classification clusters, namely, the data noise of the Internet of things, are cleaned, and then the data are cleaned.
In the embodiment of the application, the knowledge fields of the plurality of to-be-cleaned Internet of things data sets are used as the connecting terminals, each connecting terminal is connected with the adjacent connecting terminals to obtain the plurality of connecting wires, the plurality of to-be-cleaned Internet of things data sets corresponding to the connecting terminals are classified based on the probability of determining the connecting terminals and the association degree of the connecting wires, the classifying process is changed into the process of reasoning the probability of the connecting terminals and the association degree of the connecting wires, and the classifying efficiency and the classifying accuracy are improved.
As an implementation manner, the probability of each connection terminal is inferred through a preset probabilistic inference network, a knowledge field matrix corresponding to a plurality of connection terminals is obtained based on knowledge fields corresponding to the connection terminals, then a commonality measurement result matrix is obtained based on commonality measurement results between every two connection terminals, and then the knowledge field matrix and the commonality measurement result matrix are loaded into the pre-debugged probabilistic inference network to obtain the probability that the connection terminals and the adjacent connection terminals are the same classification information, wherein the probabilistic inference network can be a graph neural network.
As an implementation manner, the probabilistic inference network may be debugged through the internet of things training data set, where each internet of things training data subset in the internet of things training data set is labeled with classification information (i.e., type).
For the training data set of the Internet of things, the probability corresponding to each training data subset of the Internet of things is obtained, the probability that the training connection terminal and the adjacent connection terminal are the same in classification information is determined for each training connection terminal, and the probability is marked for the training connection terminal. And then obtaining training knowledge field matrixes corresponding to the training connection terminals based on the knowledge fields corresponding to the training connection terminals.
Based on the commonality measurement result between every two training connection terminals, a training commonality measurement result matrix is obtained, the training knowledge field matrix and the training commonality measurement result matrix are loaded into a probabilistic reasoning network, and the probability of each training connection terminal is obtained through reasoning. And adjusting network parameters of the probabilistic reasoning network based on losses between probabilities obtained by reasoning on each training connection terminal and the labeled probabilities until preset conditions are met, such as model convergence or debugging times are reached.
As an embodiment, the connection line association degree between each connection terminal and each adjacent connection terminal is inferred through the association inference network which is completed through pre-debugging. For each connection terminal, an alternative set is determined, the alternative set comprising, among the adjacent connection terminals of the connection terminals, adjacent connection terminals of the connection terminal having a probability greater than that of the connection terminal. The probability of adjacent connection terminals in the similar region is greater than that of the connection terminals, representing that the adjacent connection terminals are more prone to classifying information for one object, and in order to classify the connection terminals into characteristic classifying information, the connection terminals may be connected to the adjacent connection terminals having a higher probability than that thereof, an alternative set is determined for the connection terminals, but the selected adjacent connection terminals and connection terminals may not be the same classifying information, and consideration of the degree of association of the connection lines is required to be introduced. And then, loading the alternative set into a correlation inference network which is debugged in advance, and outputting the degree of correlation of the connecting wires between the connecting terminals and each nearest adjacent connecting terminal in the alternative set by the correlation inference network.
The correlation inference network and the probabilistic inference network may have similar structures, but the probabilistic inference network does not process the whole adjacent data set, the correlation inference network processes the sub-adjacent data set formed by the alternative set, and the correlation inference network outputs each adjacent connection terminal in the alternative set, which represents the probability that the adjacent connection terminal and the connection terminal are the same classification information.
As an implementation mode, the relevant inference network can be debugged through the internet of things training data set, wherein each internet of things training data subset in the internet of things training data set is marked with classification information. The method comprises the steps of obtaining knowledge fields of training data subsets of each Internet of things, taking the knowledge fields of the training data subsets of each Internet of things as a training connecting terminal, and obtaining training alternative sets for each training connecting terminal, wherein the probability of each alternative set comprises adjacent connecting terminals of the training connecting terminal, and the adjacent connecting terminals of the training connecting terminal are larger than that of the training alternative sets. Then connecting the training connection terminals with each adjacent connection terminal in the alternative set to obtain a training connection line, determining the association degree of the training connection line based on the classification information of the training connection terminals and the classification information of the adjacent connection terminals, marking the training connection line, determining the association degree of the connection line between the training connection terminals and the adjacent connection terminals as 1 when the classification information of the training connection terminals is consistent with the classification information of the adjacent connection terminals, and determining the association degree between the training connection terminals and the adjacent connection terminals as 0 when the classification information of the training connection terminals and the adjacent connection terminals is inconsistent; and loading the alternative set into the association inference network, inferring to obtain the association degree of each connecting wire, and adjusting network parameters of the association inference network based on the loss between the association degree of each training connecting wire inference and the marked association degree until reaching a preset condition to obtain the debugged association inference network.
Based on the same principle as the method shown in fig. 1, there is also provided a data acquisition device 10 according to an embodiment of the present application, as shown in fig. 2, the device 10 includes:
and the analysis module 11 is configured to, when an internet of things data packet uploaded by the internet of things sensor network is acquired, analyze the internet of things data packet to obtain a first internet of things data relationship network, where the first internet of things data relationship network includes a plurality of first data cluster nodes.
The mining module 12 is configured to obtain a first relationship network description vector of the first thing-connected data relationship network and first data cluster network description vectors of a plurality of first data clusters in the first thing-connected data relationship network. The first data cluster network node is a network node corresponding to data cluster information in the internet of things data packet, and the first internet of things data relation network is established based on the involvement among the plurality of first data cluster network nodes.
The data significant factor determining module 13 is configured to process the first relational network description vector and the plurality of first data cluster network node description vectors, and determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network. Wherein the data significance factor characterizes representative information of the first data cluster network in the first Internet of things data relationship network.
The screening module 14 is configured to determine a plurality of second data cluster nodes from the plurality of first data cluster nodes based on the plurality of acquired data saliency factors, where the data saliency factors corresponding to the plurality of second data cluster nodes are greater than the data saliency factors corresponding to the remaining first data cluster nodes.
And the storage classification module 15 is used for extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second internet-of-things data relation network to determine the storage classification information of the internet-of-things data packets. Wherein the second linked data relationship network is established based on involvement of the plurality of second data cluster nodes.
The above embodiment describes the data acquisition device 10 from the perspective of a virtual module, and the following describes a data acquisition cloud platform from the perspective of a physical module, specifically as follows:
an embodiment of the present application provides a data acquisition cloud platform, as shown in fig. 3, a data acquisition cloud platform 100 includes: a processor 101 and a memory 103. Wherein the processor 101 is coupled to the memory 103, such as via bus 102. Optionally, the data acquisition cloud platform 100 may also include a transceiver 104. It should be noted that, in practical application, the transceiver 104 is not limited to one, and the structure of the data collection cloud platform 100 is not limited to the embodiment of the present application.
The processor 101 may be a CPU, general purpose processor, GPU, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor 101 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
Bus 102 may include a path to transfer information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. The bus 102 may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, only one thick line is shown in fig. 3, but not only one bus or one type of bus.
Memory 103 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, optical disk storage (including compact disks, laser disks, optical disks, digital versatile disks, blu-ray disks, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The memory 103 is used for storing application program codes for executing the inventive arrangements and is controlled to be executed by the processor 101. The processor 101 is configured to execute application code stored in the memory 103 to implement what is shown in any of the method embodiments described above.
The embodiment of the application provides a data acquisition cloud platform, which comprises the following components: one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, which when executed by the one or more processors, perform the above-described data collection method based on artificial intelligence internet of things. According to the technical scheme provided by the application, the data significance factors of the first data cluster network nodes are obtained through the relational network description vector and the data cluster network node description vector, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relational network, the representative data cluster network nodes can be accurately determined in the subsequent process of selecting the first data cluster network nodes based on the reference data significance factors, and the data packet of the Internet of things is analyzed and refined based on the obtained data cluster network node description vector and the relational network description vector of the representative data cluster network node, so that the accuracy of data classification is improved, the representative data cluster network nodes are not included in the analysis category, and the efficiency of the whole storage classification flow is effectively improved.
Embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, which when run on a processor, enables the processor to perform the corresponding content of the method embodiments described above.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.

Claims (9)

1. The utility model provides a data acquisition method based on artificial intelligence thing networking, its characterized in that is applied to data acquisition cloud platform, data acquisition cloud platform and at least one thing allies oneself with sensing network communication connection, thing allies oneself with sensing network and includes thing allies oneself with relational database, thing allies oneself with relational database is used for storing thing networking data package, the method includes:
when an Internet of things data packet uploaded by the Internet of things sensing network is obtained, analyzing the Internet of things data packet to obtain a first Internet of things data relationship network, wherein the first Internet of things data relationship network comprises a plurality of first data cluster nodes;
acquiring a first relation network description vector of the first Internet of things data relation network and first data cluster network description vectors of a plurality of first data clusters in the first Internet of things data relation network; the first data cluster network node is a network node corresponding to data cluster information in the internet of things data packet, and the first internet of things data relation network is established based on the involvement among the plurality of first data cluster network nodes;
processing the first relation network description vector and a plurality of first data cluster network description vectors, and determining a data significance factor corresponding to each first data cluster network in the first Internet of things data relation network; wherein the data significance factor characterizes representative information of the first data cluster network in the first Internet of things data relationship network;
Determining a plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the plurality of acquired data significant factors, wherein the data significant factors corresponding to the plurality of second data cluster network nodes are larger than the data significant factors corresponding to the rest of first data cluster network nodes;
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes and a second relationship network description vector of a second internet-of-things data relationship network, and determining storage classification information of the internet-of-things data packets; wherein the second linked data relationship network is established based on involvement of the plurality of second data cluster nodes;
the first relation network description vector and the plurality of first data cluster network description vectors are processed, the step of determining the data significance factor corresponding to each first data cluster network in the first Internet of things data relation network is executed through a pre-debugged storage classification network, the storage classification network comprises a first data significance factor determining module, a first network node selecting module and a classification module, the first relation network description vector and the plurality of first data cluster network description vectors are processed, and the data significance factor corresponding to each first data cluster network in the first Internet of things data relation network is determined to specifically comprise:
Processing the first relation network description vector and a plurality of first data cluster network description vectors through the first data significant factor determining module, and determining the data significant factor corresponding to each first data cluster network in the first Internet of things data relation network;
the determining, based on the acquired plurality of data saliency factors, a plurality of second data cluster nodes among the plurality of first data cluster nodes includes:
determining, by the first cluster selection module, the plurality of second data cluster clusters from the plurality of first data cluster clusters based on the plurality of acquired data saliency factors;
extracting the first data cluster network node description vector of the plurality of second data cluster network nodes and the second relationship network description vector of the second link data relationship network, and determining storage classification information of the internet of things data packet, wherein the method comprises the following steps:
extracting a first data cluster network node description vector and the second relation network description vector of the plurality of second data cluster network nodes through the classifying module, and determining storage classifying information of the internet of things data packet;
the storage classification network further comprises a first linear transformation module, a second data significant factor determination module and a second network node selection module, wherein the method further comprises, after determining the plurality of second data cluster nodes in the plurality of first data cluster nodes based on the plurality of acquired data significant factors by the first network node selection module:
Correcting the first data cluster network node description vectors of the plurality of second data cluster network nodes based on the second relation network description vectors through the first linear transformation module, and determining second data cluster network node description vectors of the plurality of second data cluster network nodes;
processing the second relation network description vector and the plurality of second data cluster network description vectors through the second data significant factor determining module, and determining the data significant factor of each second data cluster network in the second internet-of-things data relation network;
and determining a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the acquired plurality of data significant factors through the second network node selection module, wherein the data significant factors corresponding to the plurality of third data cluster network nodes are larger than the data significant factors corresponding to the rest of second data cluster network nodes.
2. The method of claim 1, wherein after the step of determining a plurality of second data cluster nodes from the plurality of first data cluster nodes based on the plurality of acquired data saliency factors, the method further comprises:
correcting the first data cluster network node description vectors of the plurality of second data cluster network nodes based on the second relation network description vector to obtain second data cluster network node description vectors of the plurality of second data cluster network nodes;
Processing the second relation network description vector and the plurality of second data cluster network node description vectors, and determining a data significance factor of each second data cluster network node in the second internet-of-things data relation network;
and determining a plurality of third data cluster network nodes in the second data cluster network nodes based on the acquired plurality of data significant factors, wherein the data significant factors corresponding to the third data cluster network nodes are larger than the data significant factors corresponding to the rest second data cluster network nodes.
3. The method of claim 2, wherein extracting the first data cluster network node description vector of the plurality of second data cluster networks and the second relationship network description vector of the second linked data relationship network to determine the storage classification information of the internet of things data packet comprises:
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network node description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes and a third relation network node description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on the involvement among the plurality of third data cluster network nodes.
4. The method of claim 3, wherein the extracting the first data cluster mesh description vector of the plurality of second data cluster meshes, the second relationship mesh description vector, the second data cluster mesh description vector of the plurality of third data cluster meshes, and the third relationship mesh description vector of the third internet of things data relationship network to determine the storage classification information of the internet of things data packet comprises:
based on the numerical statistics result of the second data cluster network nodes, carrying out average value calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes, and determining data cluster network node average value description vectors corresponding to the plurality of second data cluster network nodes;
connecting the data cluster network node mean value description vector with the largest data cluster network node description vector in the first data cluster network node description vectors of the plurality of second data cluster network nodes to obtain a first merging description vector;
splicing the second data cluster network node description vectors of the plurality of third data cluster network nodes and the third relation network description vector to obtain a second merging description vector;
connecting the first merging description vector with the second merging description vector, and determining the merging description vector corresponding to the data packet of the Internet of things;
And extracting the combined description vector, and determining storage classification information of the internet of things data packet.
5. The method according to claim 1, wherein the extracting, by the classifying module, the first data cluster network description vector and the second relationship network description vector of the plurality of second data cluster networks, and determining the storage classification information of the internet of things data packet includes:
and extracting a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network node description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes and a third relation network node description vector of a third Internet of things data relation network by the classifying module, and determining storage classifying information of the Internet of things data packet, wherein the third Internet of things data relation network is established based on the involving conditions among the plurality of third data cluster network nodes.
6. The method of claim 5, wherein the storage classification network further comprises a first stitching module and a second stitching module, and wherein the determining, by the classification module, storage classification information for the internet of things data packet by extracting a first data cluster mesh description vector of the plurality of second data cluster meshes, the second relationship mesh description vector, a second data cluster mesh description vector of the plurality of third data cluster meshes, and a third relationship mesh description vector of a third internet of things data relationship network comprises:
Splicing the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors through the first splicing module to obtain a first merging description vector;
splicing the second data cluster network node description vectors of the plurality of third data cluster network nodes and the third relation network description vectors through the second splicing module to obtain second merging description vectors;
extracting the first combined description vector and the second combined description vector through the classifying module, and determining storage classifying information of the internet of things data packet;
the storage classification network further includes a merging module, and the extracting, by the classification module, the first merging description vector and the second merging description vector, to determine storage classification information of the internet of things data packet includes:
the merging module is used for connecting the first merging description vector and the second merging description vector and determining the merging description vector corresponding to the data packet of the Internet of things;
and extracting the combined description vector through the classifying module, and determining the storage classifying information of the data packet of the Internet of things.
7. The method according to any one of claims 5-6, wherein the storage classification network is obtained by debugging through the following steps:
acquiring training storage classification information of an Internet of things training data packet, training relation network description vectors of a training Internet of things data relation network and training data cluster network description vectors of a plurality of training data clusters in the training Internet of things data relation network, wherein the training data cluster network is a network corresponding to the training data cluster information, and the training Internet of things data relation network is established based on the involvement of the plurality of training data cluster information in the Internet of things training data packet;
extracting the training relation network description vector and the training data cluster network description vectors of the plurality of training data clusters through the storage classification network, and determining the reasoning storage classification information of the training data packets of the Internet of things;
and debugging network parameters of the storage classification network based on the loss between the training storage classification information and the reasoning storage classification information until a preset condition is reached, so as to obtain a debugged storage classification network.
8. The data acquisition system is characterized by comprising a data processing cloud platform and at least one Internet of things sensing network in communication connection with the data processing cloud platform, wherein the Internet of things sensing network comprises an Internet of things relational database, the Internet of things relational database is used for storing Internet of things data packets, the data processing cloud platform comprises a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, the method of any one of claims 1-7 is executed.
9. A data acquisition cloud platform comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, performs the method of any one of claims 1 to 7.
CN202211553105.0A 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform Active CN115599873B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211553105.0A CN115599873B (en) 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211553105.0A CN115599873B (en) 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform

Publications (2)

Publication Number Publication Date
CN115599873A CN115599873A (en) 2023-01-13
CN115599873B true CN115599873B (en) 2023-08-29

Family

ID=84851927

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211553105.0A Active CN115599873B (en) 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform

Country Status (1)

Country Link
CN (1) CN115599873B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117273765B (en) * 2023-11-21 2024-02-06 广州欧派创意家居设计有限公司 Multistage dealer circulation data processing method and system based on automatic check

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9311386B1 (en) * 2013-04-03 2016-04-12 Narus, Inc. Categorizing network resources and extracting user interests from network activity
WO2018219223A1 (en) * 2017-05-31 2018-12-06 腾讯科技（深圳）有限公司 Data processing method, device and storage medium
CN109033387A (en) * 2018-07-26 2018-12-18 广州大学 A kind of Internet of Things search system, method and storage medium merging multi-source data
CN109347834A (en) * 2018-10-24 2019-02-15 广东工业大学 Detection method, device and the equipment of abnormal data in Internet of Things edge calculations environment
CN109672750A (en) * 2019-01-15 2019-04-23 特斯联（北京）科技有限公司 A kind of security protection data aggregate management system and its management method based on Edge fabrics
CN110348526A (en) * 2019-07-15 2019-10-18 武汉绿色网络信息服务有限责任公司 A kind of device type recognition methods and device based on semi-supervised clustering algorithm
CN110955212A (en) * 2019-12-05 2020-04-03 河南工业大学 Wisdom agricultural information processing system based on thing networking
CN112104684A (en) * 2020-02-12 2020-12-18 云池网络科技(上海)有限公司 Platform management system based on Internet of things
CN113207101A (en) * 2021-04-13 2021-08-03 山东曙光照信息技术有限公司 Information processing method based on 5G city component sensor and Internet of things cloud platform
CN113837009A (en) * 2021-08-26 2021-12-24 张大艳 Internet of things data acquisition and analysis system based on artificial intelligence
CN114257980A (en) * 2021-12-06 2022-03-29 安徽南瑞继远电网技术有限公司 Data interaction system and interaction method based on cluster network
CN114547440A (en) * 2022-01-25 2022-05-27 王金山 User portrait mining method based on internet big data and artificial intelligence cloud system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11489914B2 (en) * 2020-01-19 2022-11-01 Bernt Erik Bjontegard Contextual intelligence of every-thing communication platform integrating edge computation, augmented intelligence and distributed ledger data storage nodes and tokens for real world data interactions across internet of things with payment integration

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9311386B1 (en) * 2013-04-03 2016-04-12 Narus, Inc. Categorizing network resources and extracting user interests from network activity
WO2018219223A1 (en) * 2017-05-31 2018-12-06 腾讯科技（深圳）有限公司 Data processing method, device and storage medium
CN109033387A (en) * 2018-07-26 2018-12-18 广州大学 A kind of Internet of Things search system, method and storage medium merging multi-source data
CN109347834A (en) * 2018-10-24 2019-02-15 广东工业大学 Detection method, device and the equipment of abnormal data in Internet of Things edge calculations environment
CN109672750A (en) * 2019-01-15 2019-04-23 特斯联（北京）科技有限公司 A kind of security protection data aggregate management system and its management method based on Edge fabrics
CN110348526A (en) * 2019-07-15 2019-10-18 武汉绿色网络信息服务有限责任公司 A kind of device type recognition methods and device based on semi-supervised clustering algorithm
CN110955212A (en) * 2019-12-05 2020-04-03 河南工业大学 Wisdom agricultural information processing system based on thing networking
CN112104684A (en) * 2020-02-12 2020-12-18 云池网络科技(上海)有限公司 Platform management system based on Internet of things
CN113207101A (en) * 2021-04-13 2021-08-03 山东曙光照信息技术有限公司 Information processing method based on 5G city component sensor and Internet of things cloud platform
CN113837009A (en) * 2021-08-26 2021-12-24 张大艳 Internet of things data acquisition and analysis system based on artificial intelligence
CN114257980A (en) * 2021-12-06 2022-03-29 安徽南瑞继远电网技术有限公司 Data interaction system and interaction method based on cluster network
CN114547440A (en) * 2022-01-25 2022-05-27 王金山 User portrait mining method based on internet big data and artificial intelligence cloud system

Also Published As

Publication number Publication date
CN115599873A (en) 2023-01-13

Similar Documents

Publication Publication Date Title
JP2021519997A (en) Heterogeneous graphs, methods for achieving identification of molecular spatial structural properties, their devices, computer devices and computer programs
CN111563192B (en) Entity alignment method, device, electronic equipment and storage medium
CN112396160A (en) Transaction fraud detection method and system based on graph neural network
CN115599873B (en) Data acquisition method and system based on artificial intelligence Internet of things and cloud platform
CN111310918A (en) Data processing method and device, computer equipment and storage medium
CN114358216B (en) Quantum clustering method based on machine learning framework and related device
CN116166642A (en) Spatio-temporal data filling method, system, equipment and medium based on guide information
CN112257332B (en) Simulation model evaluation method and device
CN113127464B (en) Agricultural big data environment feature processing method and device and electronic equipment
CN118036756B (en) Method, device, computer equipment and storage medium for large model multi-round dialogue
CN114239999A (en) Element reliability parameter optimization analysis method based on cross entropy important sampling
CN112215270B (en) Similarity comparison method, system, equipment and medium of model
CN112949778A (en) Intelligent contract classification method and system based on locality sensitive hashing and electronic equipment
CN115905924B (en) Data processing method and system based on artificial intelligence Internet of things and cloud platform
CN115544307A (en) Directed graph data feature extraction and expression method and system based on incidence matrix
CN115879501A (en) Manufacturing service data value evaluation dynamic optimization method for multi-view active learning
CN114722920A (en) Deep map convolution model phishing account identification method based on map classification
CN113449626A (en) Hidden Markov model vibration signal analysis method and device, storage medium and terminal
CN115712614B (en) Information processing method and system based on data transmission flow control
CN112104684A (en) Platform management system based on Internet of things
US12147476B2 (en) Embedding and analyzing multivariate information in graph structures
CN117041121B (en) Internet of Things anomaly monitoring method and system based on data mining
CN111563033B (en) Simulation data generation method and device
CN117808081A (en) Deep neural network distributed training method based on model structure automatic analysis
CN117371509A (en) Classification method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20230807

Address after: 510000 room 6-211, Part 5, 6 and 7, No. 67, Dongpu Second Road, Tianhe District, Guangzhou City, Guangdong Province (office only)

Applicant after: Guangzhou Fengwang Internet Technology Co.,Ltd.

Address before: No. 109, Ankang Street, Bincheng District, Binzhou City, Shandong Province, 256600 - 0085

Applicant before: Binzhou Yuwei Network Technology Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant