CN115599873A - Data acquisition method and system based on artificial intelligence Internet of things and cloud platform - Google Patents

Data acquisition method and system based on artificial intelligence Internet of things and cloud platform Download PDF

Info

Publication number
CN115599873A
CN115599873A CN202211553105.0A CN202211553105A CN115599873A CN 115599873 A CN115599873 A CN 115599873A CN 202211553105 A CN202211553105 A CN 202211553105A CN 115599873 A CN115599873 A CN 115599873A
Authority
CN
China
Prior art keywords
data
network
internet
things
data cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211553105.0A
Other languages
Chinese (zh)
Other versions
CN115599873B (en
Inventor
陈城
陈劲峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Fengwang Internet Technology Co ltd
Original Assignee
Binzhou Yuwei Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Binzhou Yuwei Network Technology Co ltd filed Critical Binzhou Yuwei Network Technology Co ltd
Priority to CN202211553105.0A priority Critical patent/CN115599873B/en
Publication of CN115599873A publication Critical patent/CN115599873A/en
Application granted granted Critical
Publication of CN115599873B publication Critical patent/CN115599873B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/285Clustering or classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Abstract

According to the data acquisition method, system and cloud platform based on the artificial intelligence Internet of things, the relation network description vectors and the data cluster network node description vectors are used for obtaining the data significance factors of each first data cluster network node, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relation network, the representative data cluster network nodes can be accurately determined in the process of selecting each first data cluster network node based on the reference data significance factors, the Internet of things data packets are analyzed and refined based on the obtained data cluster network node description vectors and the relation network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification flow is effectively improved.

Description

Data acquisition method and system based on artificial intelligence Internet of things and cloud platform
Technical Field
The application relates to the technical field of artificial intelligence and the Internet of things, in particular to a data acquisition method and system based on the artificial intelligence Internet of things and a cloud platform.
Background
With the wider application of artificial intelligence and the internet of things, the internet of things equipment generates a large amount of data, and the data are analyzed and tracked by artificial intelligence and machine learning. In this way, artificial intelligence is combined with the internet of things to create intelligent devices and make informed decisions without human intervention. The possibilities offered by the internet of things are unlimited, and the rapid expansion of networked devices and sensors makes the amount of data they create exponentially growing, and the problem that how to analyze these massive performance data is obviously unrealistic by human power, while machine learning in artificial intelligence is an effective solution. Before analyzing the data, the data acquisition is a precondition, and in the data acquisition process, the data needs to be classified and stored so as to be conveniently and subsequently extracted in a targeted manner according to the analysis requirement.
Disclosure of Invention
The invention aims to provide a data acquisition method, a data acquisition system and a cloud platform based on an artificial intelligence Internet of things so as to improve the efficiency of data classification and storage.
The embodiment of the application achieves the aim as follows:
in a first aspect, an embodiment of the present application provides a data acquisition method based on an artificial intelligence internet of things, which is characterized in that the data acquisition method is applied to a data acquisition cloud platform, the data acquisition cloud platform is in communication connection with at least one internet of things sensor network, the internet of things sensor network includes an internet of things relational database, the internet of things relational database is used for storing internet of things data packages, and the method includes:
when an internet of things data packet uploaded by the internet of things sensor network is acquired, analyzing the internet of things data packet to acquire a first internet of things data relation network, wherein the first internet of things data relation network comprises a plurality of first data cluster network nodes;
acquiring a first relation network description vector of the first internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first internet of things data relation network; the first data cluster network nodes are network nodes corresponding to data cluster information in the data packet of the Internet of things, and the first Internet of things data relation network is established based on the involvement conditions among the plurality of first data cluster network nodes;
processing the first relational network description vector and the plurality of first data cluster network node description vectors to determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network; wherein the data significance factor characterizes representative information of the first data cluster network node in the first Internet of things data relationship network;
determining a plurality of second data clustering nodes in the plurality of first data clustering nodes based on the obtained plurality of data saliency factors, wherein the data saliency factors corresponding to the plurality of second data clustering nodes are larger than the data saliency factors corresponding to the rest of the first data clustering nodes;
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes and a second relation network description vector of a second relation data relation network to determine storage classification information of the data packet of the internet of things; wherein the second associative data relationship network is established based on the involvement between the plurality of second data cluster network nodes.
As a possible implementation manner, after the step of determining a plurality of second data cluster nodes in the plurality of first data cluster nodes based on the obtained plurality of data saliency factors, the method further includes:
modifying a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relational network description vector to obtain a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors to determine a data significance factor of each second data cluster network node in the second relational data network;
determining a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the obtained plurality of data significance factors, wherein the data significance factors corresponding to the plurality of third data cluster network nodes are larger than the data significance factors corresponding to the rest of second data cluster network nodes.
As a possible implementation manner, the extracting a first data cluster node description vector of the plurality of second data cluster nodes and a second data relationship network description vector of a second data relationship network to determine storage classification information of the internet of things data packet includes:
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement between the plurality of third data cluster network nodes.
As a possible implementation manner, the extracting the first data cluster node description vector of the second data cluster nodes, the second relation network description vector, the second data cluster node description vector of the third data cluster nodes, and the third relation network description vector of the third internet of things node to determine the storage classification information of the internet of things packet includes:
based on the numerical statistical result of the second data cluster network nodes, performing mean calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes to determine data cluster network node mean description vectors corresponding to the plurality of second data cluster network nodes;
connecting the data cluster network node mean description vector with a maximum data cluster network node description vector in a first data cluster network node description vector of the plurality of second data cluster network nodes to obtain a first combined description vector;
splicing second data clustering network node description vectors of the plurality of third data clustering network nodes and the third relation network description vector to obtain a second combined description vector;
connecting the first merging description vector with the second merging description vector, and determining a merging description vector corresponding to the Internet of things data packet;
and extracting the merged description vector, and determining storage classification information of the Internet of things data packet.
As a possible implementation manner, the step of processing the first relational network description vector and the plurality of first data cluster network description vectors by the data saliency determination module, and determining the data saliency corresponding to each first data cluster network junction in the first internet of things data relational network is performed by a storage classification network debugged in advance, where the storage classification network includes a first data saliency determination module, a first network junction selection module, and a classification module, and the step of processing the first relational network description vector and the plurality of first data cluster network junction description vectors, and determining the data saliency corresponding to each first data cluster network junction in the first internet of things data relational network specifically includes:
processing the first relational network description vector and the plurality of first data cluster network node description vectors through the first data significance factor determining module, and determining a data significance factor corresponding to each first data cluster network node in the first internet of things data relational network;
the determining a plurality of second data cluster nodes among the plurality of first data cluster nodes based on the obtained plurality of data saliency factors includes:
determining, by the first network node selection module, the plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the obtained plurality of data saliency factors;
the extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second relation data relation network to determine the storage classification information of the internet of things data packet includes:
and extracting the first data clustering network node description vector and the second relation network description vector of the plurality of second data clustering network nodes through the classification module, and determining the storage classification information of the data packet of the internet of things.
The storage and classification network further includes a first linear transformation module, a second data saliency factor determination module, and a second network node selection module, and after the first network node selection module determines the plurality of second data clustering network nodes among the plurality of first data clustering network nodes based on the obtained plurality of data saliency factors, the method further includes:
modifying, by the first linear transformation module, a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relationship network description vector, and determining a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors through the second data significant factor determination module to determine a data significant factor of each second data cluster network node in the second relational network;
determining, by the second network node selection module, a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the obtained plurality of data significance factors, where the data significance factors corresponding to the plurality of third data cluster network nodes are greater than the data significance factors corresponding to the remaining second data cluster network nodes.
As a possible implementation manner, the extracting, by the classification module, the first data cluster node description vector and the second relationship network description vector of the plurality of second data cluster nodes to determine storage classification information of the internet of things data packet includes:
extracting, by the classification module, a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes, and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement conditions among the plurality of third data cluster network nodes.
As a possible implementation manner, the storage classification network further includes a first splicing module and a second splicing module, and the determining, by the classification module, storage classification information of the internet of things data packet by extracting a first data clustering network node description vector of the second data clustering network nodes, the second relation network description vector, a second data clustering network node description vector of the third data clustering network nodes, and a third relation network description vector of a third internet of things data relation network includes:
splicing a first data cluster network node description vector and a second relation network description vector of the plurality of second data cluster network nodes through the first splicing module to obtain a first combined description vector;
splicing a second data cluster network node description vector and the third relation network description vector of the plurality of third data cluster network nodes through the second splicing module to obtain a second combined description vector;
extracting the first merging description vector and the second merging description vector through the classification module, and determining storage classification information of the data packet of the Internet of things;
the storage classification network further comprises a merging module, and the extracting of the first merging description vector and the second merging description vector is performed through the classification module to determine the storage classification information of the data packet of the internet of things, wherein the merging module comprises:
connecting the first merging description vector and the second merging description vector through the merging module, and determining a merging description vector corresponding to the data packet of the internet of things;
and extracting the merged description vector through the classification module, and determining the storage classification information of the data packet of the Internet of things.
As a possible implementation manner, the storage classification network is obtained by debugging the following steps:
acquiring training storage classification information of an Internet of things training data packet, a training relation network description vector of a training Internet of things data relation network and a training data cluster network node description vector of a plurality of training data cluster network nodes in the training Internet of things data relation network, wherein the training data cluster network node is a network node corresponding to the training data cluster information, and the training Internet of things data relation network is established based on the involvement condition among the plurality of training data cluster information in the Internet of things training data packet;
extracting the training relation network description vector and the training data clustering network node description vectors of the training data clustering network nodes through the storage classification network, and determining inference storage classification information of the training data packet of the Internet of things;
and debugging the network parameters of the storage classification network until reaching a preset condition based on the loss between the training storage classification information and the reasoning storage classification information to obtain the storage classification network after debugging.
In a second aspect, an embodiment of the present application provides a data acquisition system, including a data processing cloud platform and at least one internet of things sensor network in communication connection with the data processing cloud platform, the internet of things sensor network includes an internet of things relational database, the internet of things relational database is used for storing internet of things data packets, the data processing cloud platform includes a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, the method described above is performed.
In a third aspect, an embodiment of the present application provides a data acquisition cloud platform, which includes a processor and a memory, where the memory stores a computer program, and when the processor executes the computer program, the method described above is performed.
According to the data acquisition method, the system and the cloud platform based on the artificial intelligence Internet of things, the relation network description vectors and the data cluster network node description vectors are used for obtaining the data significance factors of each first data cluster network node, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relation network, the representative data cluster network nodes can be accurately determined in the process of selecting each first data cluster network node based on the reference data significance factors, the data packet of the Internet of things can be analyzed and refined based on the obtained data cluster network node description vectors and the relation network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification flow is effectively improved.
In the description that follows, additional features will be set forth, in part, in the description. These features will be in part apparent to those skilled in the art upon examination of the following and the accompanying drawings, or may be learned by production or use. The features of the present application may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities and combinations particularly pointed out in the detailed examples which follow.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a flowchart of a data acquisition method based on an artificial intelligence internet of things according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a functional module architecture of a data acquisition device according to an embodiment of the present application.
Fig. 3 is a schematic composition diagram of a data acquisition cloud platform provided in an embodiment of the present application.
Detailed Description
The embodiments of the present application will be described below with reference to the drawings. The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments herein only and is not intended to be limiting of the application.
In the embodiment of the application, an execution subject of the data acquisition method based on the artificial intelligence internet of things is a data acquisition cloud platform, including but not limited to a network server, a server group consisting of a plurality of network servers, or a cloud consisting of a large number of computers or network servers in cloud computing, wherein the cloud computing is one of distributed computing and is a super virtual computer consisting of a group of loosely coupled computers. The data acquisition cloud platform can operate independently to achieve the application, and can also be accessed to a network and achieve the application through interactive operation with other data acquisition cloud platforms in the network. The network where the data acquisition cloud platform is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network and the like; the data acquisition cloud platform is in communication connection with at least one Internet of things sensing network, the Internet of things sensing network comprises an Internet of things relation database, the Internet of things relation database is used for storing Internet of things data packets, the Internet of things sensing network senses and arranges the Internet of things data packets according to a fixed communication protocol through sensors distributed in a target area, the arranged Internet of things data packets are stored in the Internet of things relation database and then sent to the data acquisition cloud platform, and the Internet of things data packets sent by the Internet of things sensing network can be sent in real time or non-real time, such as regular (sent according to a preset period) or quantitative (sent when the data volume reaches a preset size) sending mode. The data in the data packet of the internet of things may be static data or dynamic data, and may be energy type data (related to energy consumption or related data required for calculating energy consumption), asset attribute type data (hardware asset data), diagnostic type data (data for detecting an operation state of the device during operation), and signal type data. The data acquisition cloud platform and the Internet of things sensing network jointly form a data acquisition system.
The embodiment of the application provides a data acquisition method based on an artificial intelligence internet of things, which is applied to a data acquisition cloud platform and comprises the following steps:
step S100: when the Internet of things data packet uploaded by the Internet of things sensor network is obtained, the Internet of things data packet is analyzed to obtain a first Internet of things data relation network, and the first Internet of things data relation network comprises a plurality of first data cluster network nodes.
In the embodiment of the application, the internet of things data packets uploaded by the internet of things sensor network are data packets needing to be classified and stored, namely, the type of the attribution of the internet of things data packets needs to be determined and then stored in the storage space of the same type of data, so that subsequent data processing is facilitated, and the data acquisition process is completed. The data packet of the internet of things comprises a plurality of data clusters, for example, the data of the internet of things contained in each data cluster is the data sensed by the corresponding sensor, and the data packet of the internet of things is formed by each data cluster. In the internet of things data packets formed by the data clusters, the classification contributions of different data clusters to the whole internet of things data packet are different, for example, the data volume proportion, the preset data requirement satisfaction condition and the like.
Step S200: the method comprises the steps of obtaining a first relation network description vector of a first Internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first Internet of things data relation network.
The first internet of things data relation network is established based on the involvement between a plurality of first data cluster network nodes, for example, an internet of things data packet comprises data clusters 1 to 10, wherein the data cluster 1 contains type A sensor data, the data cluster 2 contains type B sensor data, the data cluster 3 contains time sequence data, the data cluster 4 contains coordinate data 8230, the data cluster 8230, wherein the time stamps of the data in the data cluster 1 and the data cluster 2 are recorded in the data cluster 3, the coordinate information is recorded in the data cluster 4, the data cluster 1 and the data cluster 3 and the data cluster 4 are involved (can generate correlation), the data cluster 2 and the data cluster 3 and the data cluster 4 are involved, and the data cluster 1 and the data cluster 2 can be involved or not involved. If the two data clusters involve, the two data clusters are connected, in other words, the first data network nodes corresponding to the data clusters are connected, and finally the first Internet of things data relation network is obtained. The first data cluster network node is a network node corresponding to data cluster information in the data packet of the internet of things, and the data cluster information is information indicating why the data cluster is a data cluster, and may be, for example, a tag or a numerical value. As an implementation manner, if the plurality of first data cluster nodes are data cluster nodes corresponding to all data cluster information in the data packet of the internet of things, in other words, the plurality of first data cluster nodes are not selected, the first internet of things data relationship network is an original internet of things data relationship network, and if the plurality of first data cluster nodes are data cluster nodes corresponding to part of data cluster information in the data packet of the internet of things, in other words, the plurality of first data cluster nodes are already selected, the first internet of things data relationship network is a selected internet of things data relationship network. In the embodiment of the present application, a first relational network description vector (a vector expression embodying features) is used to describe a first internet of things data relational network, and a first data cluster network node description vector is used to describe a data cluster corresponding to data cluster information, for example, the first data cluster network node description vector includes information content of the data cluster and attributes of data of the data cluster.
Step S300: and processing the first relation network description vector and the plurality of first data cluster network node description vectors to obtain a data significance factor corresponding to each first data cluster network node in the first Internet of things data relation network.
The data significance factor represents representative information of the first data cluster network node in the first internet of things data relationship network, in other words, it indicates representative information of data cluster information corresponding to the first data cluster network node in an internet of things data packet, or importance, the data significance factor can be expressed in a numerical manner, the representative information of the data cluster information in the internet of things data packet is positively correlated with the data significance factor, that is, the larger the data significance factor is, the stronger the representative information is, for example, a plurality of sensors arranged in the same bit field, the most important core sensor is for analysis purposes, the representative of the generated data cluster is the highest, and the greater the significance factor corresponding to the data cluster is.
Step S400: and determining a plurality of second data clustering nodes from the plurality of first data clustering nodes based on the obtained plurality of data saliency factors.
The data significance factor corresponding to the plurality of second data clustering nodes is greater than the data significance factor corresponding to the rest of the first data clustering nodes. In step 400, a plurality of first data network nodes are selected, a plurality of corresponding second data network nodes with a larger data significance factor (for example, reaching a preset value) are determined, and then processing is performed based on the selected second data network nodes with a larger data significance factor, for example, information with poor representativeness is cleaned, so that data to be processed is reduced, and representative data is highlighted.
Step S500: and extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second relation data relation network to obtain the storage classification information of the data packet of the Internet of things.
The second data relation network is established based on the involvement between the plurality of second data cluster network nodes, and it can be correspondingly understood that the second relation network description vector can describe the features of the second data relation network, the second data relation network is a part of the first data relation network, the involvement between the plurality of second data cluster network nodes and the plurality of second data cluster network nodes covered in the second data relation network, and the connection between the plurality of second data cluster network nodes in the second data relation network is consistent with the connection between the plurality of second data cluster network nodes in the first data relation network.
The storage classification information represents classification information corresponding to a storage space in which the data packet of the internet of things needs to be stored, and the obtained storage classification information is different through preset classification elements and classification conditions. For example, the preset classification elements are divided according to regions, so that the acquisition regions can be classified according to region information corresponding to the data cluster information in the data packet of the internet of things, and the preset classification elements are divided according to a numerical range, so that the acquisition numerical values are classified according to a numerical interval corresponding to the data cluster information in the data packet of the internet of things.
In the steps S100 to S500, the data significant factor of each first data cluster network node is obtained through the relationship network description vector and the data cluster network node description vector, richer reference information is covered, and the obtained data significant factor can fully reflect the representativeness of the data cluster network node in the internet of things data relationship network, so that a representative data cluster network node can be accurately determined in the subsequent process of selecting each first data cluster network node based on the reference data significant factor, and the internet of things data packet is analyzed and refined based on the obtained data cluster network node description vector and the relationship network description vector of the representative data cluster network node, so that the accuracy of data classification is increased, the data cluster network node with poor representativeness is not included in the category analysis, and the efficiency of the whole storage classification process is effectively improved.
As an implementation manner, the data processing cloud platform stores and classifies the internet of things data packets through a storage and classification network, the storage and classification network may be any feasible artificial intelligence network, such as a deep learning neural network, and specifically may include a data loading module, a first data significant factor determining module, a first net knot selecting module and a classification module, wherein the data loading module is connected with the first data significant factor determining module, the first data significant factor determining module is connected with the first net knot selecting module, and the first net knot selecting module is connected with the classification module. The data loading module is configured to obtain input relational network description vectors and data clustering network node description vectors, the first data saliency factor determination module is configured to obtain a data saliency factor of each data clustering network node, the first network node selection module is configured to determine a representative data clustering network node based on the obtained data saliency factor, and the classification module is configured to determine storage classification information based on the determined data clustering network node description vectors of the data clustering network nodes and the corresponding relational network description vectors.
As an implementation manner, the storage and classification network further includes a first linear transformation module, a second data significant factor determination module, a second network node selection module and a merging module, wherein the first linear transformation module is connected to the first network node selection module and the second data significant factor determination module, the second data significant factor determination module is connected to the second network node selection module, the second network node selection module is connected to the classification module, and the merging module is connected to the first network node selection module, the second network node selection module and the classification module. The first linear transformation module is configured to perform deep processing on data cluster network node description vectors of representative data cluster network nodes determined by the first network node selection module, the deep processing of the linear transformation module can be convolution operation, the second data significance factor determination module is configured to obtain a data significance factor of each determined data cluster network node, the second network node selection module is configured to perform further screening on the determined data cluster network nodes according to the obtained data significance factors, and the combination module is configured to connect the data cluster network node description vectors of the data cluster network nodes determined by the first network node selection module and the data cluster network node description vectors of the data cluster network nodes determined again by the second network node selection module. It can be reasoned that in other embodiments, more filtering picks can be made on a data cluster to get more richness of information. The number of the corresponding data saliency determination modules and network node selection modules becomes larger, and the data cluster network node description vectors of the data cluster network nodes determined by more network node selection modules are connected through the merging module.
As an implementation manner, a linear transformation module may be arranged before the first data significant factor determination module, and configured to perform convolution operation on the input data cluster network structure description vector and the relationship network description vector, where the first data significant factor determination module determines the data significant factor according to the convolved data cluster network structure description vector and the relationship network description vector.
Based on the network architecture, the data acquisition method based on the artificial intelligence internet of things provided by the embodiment of the application can comprise the following steps:
step S10: and establishing a first Internet of things data relation network according to the involvement condition among a plurality of data cluster information in the Internet of things data packet.
Taking an example that a first internet of things data relationship network is an internet of things data relationship network established first as an illustration, in the embodiment of the present application, corresponding data cluster network nodes are established based on a plurality of data cluster information included in an internet of things data packet, and then, the data cluster network nodes are connected based on involvement between the plurality of data cluster information, so as to weave a first internet of things data relationship network, where the first internet of things data relationship network includes only a single type of network node of the data cluster network node, and in addition, the data cluster network nodes are connected based on the same type of involvement, in other words, the connection relationship in the first internet of things data relationship network is a consistent connection relationship, where the involvement between two random data cluster network nodes depends on an involvement degree vector between the two data cluster network nodes, and the involvement degree vector represents a degree of correlation between data clusters corresponding to the two data cluster network nodes, for example, the degree of correlation between the data cluster 1 and the data cluster 2 is low, and the degree of correlation between the data cluster 1 and the data cluster 3 is high.
S20: the method comprises the steps of obtaining a first relation network description vector of a first Internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first Internet of things data relation network.
The first relational network description vector represents a plurality of data cluster network nodes in a first internet of things data relational network and the involvement condition between the data cluster network nodes, the first relational network description vector comprises a correlation degree vector between two random data cluster network nodes in the data cluster network nodes, the first data cluster network node description vector comprises an information content description vector of a data cluster and an attribute description vector of data cluster data, the information content description vector of the data cluster represents the data expression meaning of the data cluster, and the attribute description vector of the data cluster data represents the data attribute information of the data cluster, such as data format and data size, and the information content description vector and the attribute description vector of the data cluster data can extract feature vectors in a universal feature extraction mode.
As an implementation manner, a plurality of first data cluster network node description vectors may be connected to form a matrix, where the content in the vertical and horizontal relationships represents the first data cluster network node description vectors of the first data cluster network nodes, and similarly, the first relationship network description vectors may also form a matrix, where each value represents the involvement of two data cluster network nodes.
S30: and processing the first relation network description vector and the plurality of first data cluster network node description vectors through a first data significant factor determining module to obtain a data significant factor corresponding to each first data cluster network node in the first Internet of things data relation network.
The first data significant factor determining module may be a module formed by a neural network of a graph, and the process of obtaining the data significant factor may refer to a general local output function (local output function).
S40: and determining a plurality of second data cluster network nodes from the plurality of first data cluster network nodes based on the obtained plurality of data significant factors through a first network node selection module, and establishing a second associated data relation network based on the involvement between the plurality of second data cluster network nodes.
The data significance factor corresponding to a plurality of second data clustering network nodes is larger than the data significance factor corresponding to the rest first data clustering network nodes. As an implementation manner, a first network node selection module multiplies the numerical statistical results of a plurality of first data cluster network nodes by a preset parameter to obtain reference numerical statistical results, arranges the data significant factors of the plurality of first data cluster network nodes according to the numerical values, determines the data significant factors of the plurality of reference numerical statistical results with larger numerical values, determines the first data cluster network nodes corresponding to the determined data significant factors as a plurality of obtained second data cluster network nodes, and establishes a second associated data relationship network based on the obtained involvement between the plurality of second data cluster network nodes after the plurality of second data cluster network nodes are obtained.
Based on consideration of the data significance factor, the first data cluster network node description vectors of the plurality of second data cluster network nodes may be modified to obtain modified first data cluster network node description vectors, for example, the first data cluster network node description vectors of the second data cluster network nodes are multiplied by the corresponding data significance factor to obtain modified first data cluster network node description vectors.
The above process is explained based on multiple selections of the data cluster nodes, and if the data cluster nodes are selected only once, the storage classification information of the data packet of the internet of things can be obtained directly based on the data cluster node description vectors of the second data cluster nodes and the second relation network description vectors for establishing the second relation data relation network after the second data cluster nodes are determined.
S50: and modifying the first data cluster network node description vectors of the plurality of second data cluster network nodes through the first linear transformation module based on the second relation network description vectors to obtain the second data cluster network node description vectors of the plurality of second data cluster network nodes.
The second data cluster network junction description vector may be obtained by multiplying several matrices, i.e. multiplying the second relation network description vector, the first data cluster network junction description vector of the second data cluster network junction and the parameters of the first linear transformation module.
S60: and processing the second relational network description vector and the plurality of second data cluster network node description vectors through a second data significant factor determining module to obtain a data significant factor of each second data cluster network node in the second relational network.
S70: and determining a plurality of third data cluster network nodes from the plurality of second data cluster network nodes through a second network node selection module based on the obtained plurality of data significant factors, and establishing a third internet of things data relationship network according to the involvement conditions among the plurality of third data cluster network nodes.
S80: and extracting the first data cluster network node description vectors, the second relation network description vectors, the second data cluster network node description vectors and the third relation network description vectors of the third data cluster network nodes of the plurality of second data cluster network nodes through the classification module to obtain the storage classification information of the data packet of the Internet of things.
As an implementation manner, based on consideration of pressure of processing data by the reduction classification module, a first data cluster network node description vector and a second relation network description vector of a plurality of second data cluster network nodes may be spliced to obtain a first merged description vector, and a second data cluster network node description vector and a third relation network description vector of a plurality of third data cluster network nodes may be spliced to obtain a second merged description vector; and extracting the first merging description vector and the second merging description vector through a classification module to obtain storage classification information of the data packet of the Internet of things.
As an implementation manner, based on the numerical statistical result of the second data cluster network nodes, performing mean calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes to obtain data cluster network node mean description vectors corresponding to the plurality of second data cluster network nodes, and connecting the data cluster network mean description vectors with the largest data cluster network node description vector in the first data cluster network node description vectors of the plurality of second data cluster network nodes to obtain first combined description vectors. And for a second data cluster network node description vector and a third relation network description vector of a third data cluster network node, performing mean value calculation on a second data cluster network node description vector and a third relation network description vector of a plurality of third data cluster network nodes according to a numerical value statistical result of the third data cluster network node to obtain a data cluster network node mean value description vector corresponding to the plurality of third data cluster network nodes, and connecting the data cluster network node mean value description vector with a maximum data cluster network node description vector in the second data cluster network node description vectors of the plurality of third data cluster network nodes to obtain a second combined description vector.
As an implementation manner, the first merged description vector and the second merged description vector are connected to obtain a merged description vector corresponding to the internet of things data packet, and the merged description vector is extracted to obtain storage classification information corresponding to the internet of things data packet. The merged description vector is the feature representing the data packet of the internet of things.
As an implementation mode, the storage classification network further comprises a first splicing module, a second splicing module and a merging module, wherein the first splicing module splices a first data clustering network node description vector and a second relation network description vector of a plurality of second data clustering network nodes to obtain a first merging description vector, the second splicing module splices a second data clustering network node description vector and a third relation network description vector of a plurality of third data clustering network nodes to obtain a second merging description vector, and the merging module connects the first merging description vector and the second merging description vector to obtain a merging description vector corresponding to the data packet of the internet of things; and extracting the combined description vector through a classification module to obtain storage classification information of the data packet of the Internet of things.
According to the data acquisition method based on the artificial intelligence Internet of things, the data significance factors of the first data cluster network nodes are obtained through the relation network description vectors and the data cluster network node description vectors, richer reference information is covered, the representativeness of the data cluster network nodes in the Internet of things data relation network can be fully reflected by the obtained data significance factors, the representative data cluster network nodes can be accurately determined in the process of selecting the first data cluster network nodes based on the reference data significance factors, the Internet of things data packets are analyzed and refined based on the obtained data cluster network node description vectors and the relation network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification process is effectively improved. In addition, multiple selections are performed on the data clustering network nodes, data clustering network node description vectors and relation network description vectors of different levels are obtained, and when the data packet of the Internet of things is analyzed, the data clustering network node description vectors and the relation network description vectors of different levels are considered, so that the classification accuracy is further improved.
As an implementation manner, the embodiment of the present application further provides a debugging process of the storage classification network.
The debugging process of the storage classification network comprises the following steps: acquiring training storage classification information of an Internet of things training data packet, training relation network description vectors of a training Internet of things data relation network and training data cluster network node description vectors of a plurality of training data cluster network nodes in the training Internet of things data relation network; extracting the training relation network description vector and the training data cluster network node description vectors of the training data cluster network nodes through a storage classification network to obtain inference storage classification information of the training data packet of the Internet of things, and debugging network parameters of the storage classification network until a preset condition is met based on loss between the training storage classification information and the inference storage classification information to obtain the storage classification network after debugging. The training data cluster network node is a network node corresponding to training data cluster information, the training internet of things data relation network is established based on the involvement condition among a plurality of pieces of training data cluster information in the internet of things training data packet, and the preset condition can be that the debugging turn reaches preset times or the inference precision of the network reaches preset precision.
In order to improve the inference precision of the network, after the storage classification information of the data packet of the internet of things is analyzed through the storage classification network, the storage classification network can be further debugged according to the data packet of the internet of things.
As an implementation manner, after the internet of things data packet is stored and classified, a process of verifying integrity and correctness of the internet of things data, that is, a process of cleaning the internet of things data, may be further included, where the process includes the following steps:
step S600: the method comprises the steps of obtaining knowledge fields of a plurality of data sets of the internet of things to be cleaned, and determining the knowledge field of each data set of the internet of things to be cleaned as a connecting terminal.
The data sets of the internet of things to be cleaned can be a set formed by classified data of the internet of things, a plurality of data sets of the internet of things to be cleaned can be loaded into a convolutional neural network debugged in advance as a whole data set of the internet of things to be cleaned, and a knowledge field of each data set of the internet of things to be cleaned is acquired and used for representing characteristic information of the data sets of the internet of things.
Step S700: each connection terminal is connected to an adjacent connection terminal.
The neighboring connection terminals of the connection terminals may be obtained based on a commonality measurement result (representing a degree of similarity between the connection terminals), for example, calculating a vector distance between the connection terminals, the closer the distance, the higher the commonality measurement result. For example, computing the cosine distance between vectors. As an embodiment, for each connection terminal, M connection terminals with the largest result of the measure of commonality with the connection terminal are obtained as the adjacent connection terminals of the connection terminal, M is selected depending on the amount of data to be classified, each connection terminal is connected to the corresponding adjacent connection terminal, and then a plurality of connection lines are obtained, one of which connects two connection terminals.
Step S800: and acquiring the probability of the same classification information of the connecting terminal and the adjacent connecting terminal.
In the embodiment of the present application, the probability refers to the possibility that a connection terminal belongs to a target classification information, and it is evaluated whether the connection terminal and the similar region are sufficiently similar, and the similar region represents the region generated by the adjacent connection terminal of each connection terminal, and is the same classification information. For example, a connection terminal with a high probability determines that there are other connection terminals and that they are the same classification information, in other words, a connection terminal with a high probability has a high probability of being a classification information for itself, whereas a connection terminal with a low probability cannot determine whether there is a connection terminal and that it is the same classification information, in other words, a connection terminal with a low probability has a low probability of being a target classification information for itself.
As an embodiment, the probability that the connection terminal and the adjacent connection terminal are the same classification information is a difference between a first result of the commonality measurement that is a sum of results of the commonality measurements that the adjacent connection terminal and the connection terminal of the connection terminal are the same classification information and a second result of the commonality measurement that the adjacent connection terminal and the connection terminal of the connection terminal are a sum of results of the commonality measurements that are different classification information. For connection terminals for which classification information cannot be determined, a probabilistic inference network can be implemented to infer the probability of the connection terminal based on the completion of the debugging.
Step S900: the degree of association of the connection line between the connection terminal and each adjacent connection terminal is determined based on the likelihood of the two connection terminals being connected being the same classification information.
And if the connecting lines with strong association degree reflect that the possibility that the two connected connecting terminals are the same classification information is high, the association degree of the connecting lines with the possibility that the two connected connecting terminals are the same classification information being 1 is set as 1, otherwise, the association degree of the connecting lines with the possibility that the two connected connecting terminals are the same classification information being 0 is set as 0, the association degree of the connecting lines between the two connecting terminals which cannot determine the classification information is 0 to 1, and the association degree of the connecting lines is deduced through an association reasoning network which is debugged in advance.
Step S1000: and based on the probability of each connecting terminal and the association degree of each connecting wire, classifying the plurality of connecting terminals which are mutually mapped with the plurality of Internet of things data sets to be cleaned to obtain classification clusters of the plurality of Internet of things data sets to be cleaned.
For the adjacent data sets for determining the probability of the connecting terminals and the association degree of the connecting lines, a plurality of networking data sets to be cleaned corresponding to the connecting terminals can be classified (namely clustered) through a preset classification algorithm.
In one embodiment, a connection line between a connection terminal and a target adjacent connection terminal may be obtained, the adjacent connection terminal including the target adjacent connection terminal, and further, a probability of the target adjacent connection terminal is greater than that of the connection terminal and the connection line between the target adjacent connection terminal and the connection terminal includes a maximum degree of association. In other words, for each connection terminal, the connection line obtained is generated between the adjacent connection terminal having a probability greater than that of the connection terminal and the connection terminal, and the connection line includes the greatest degree of association among the connection lines generated by all the adjacent connection terminals and the connection terminals. Classifying the data set of the to-be-cleaned internet of things corresponding to the connecting terminal based on a through interval generated by the connecting line between the connecting terminal and the target adjacent connecting terminal to obtain at least one connecting terminal classification set, in other words, at least one cluster. Each through interval represents classification information, or each connection terminal classification set corresponds to the classification information, and classification clusters of a plurality of Internet of things data sets to be cleaned can be obtained based on at least one connection terminal classification set.
Step S1100: and cleaning the data of the Internet of things which are dissociated outside each classification cluster.
And removing the Internet of things data outside each classification cluster, namely the isolated points among the classification clusters, namely the Internet of things data noise, and then finishing the cleaning of the data.
In the embodiment of the application, the knowledge fields of a plurality of Internet of things data sets to be cleaned are used as the connecting terminals, each connecting terminal is connected with the adjacent connecting terminals to obtain a plurality of connecting wires, based on the probability of determining the connecting terminals and the association degree of the connecting wires, a plurality of Internet of things data sets to be cleaned corresponding to the connecting terminals are classified, the classifying process is changed into a process for reasoning the probability of the connecting terminals and the association degree of the connecting wires, and the classifying efficiency and accuracy are improved.
As an implementation mode, the probability of each connecting terminal is inferred through a preset probabilistic inference network, a knowledge field matrix corresponding to a plurality of connecting terminals is obtained on the basis of knowledge fields corresponding to the connecting terminals, then a commonality measurement result matrix is obtained on the basis of a commonality measurement result between every two connecting terminals, and then the knowledge field matrix and the commonality measurement result matrix are loaded to the probabilistic inference network debugged in advance to obtain the probability that the connecting terminals and adjacent connecting terminals are the same classification information, wherein the probabilistic inference network can be a graph neural network.
As an embodiment, the probabilistic inference network may be debugged through an internet of things training data set, where each internet of things training data subset in the internet of things training data set is labeled with classification information (i.e., type).
For the training data set of the internet of things, the probability corresponding to each training data subset of the internet of things is obtained, the contents can be referred to, for the training data set of the internet of things, the knowledge field of each training data subset of the internet of things is obtained, the knowledge field of each training data subset of the internet of things is used as a training connecting terminal, for each training connecting terminal, the probability that the training connecting terminal and the adjacent connecting terminal are the same classification information is determined, and the probability is marked for the training connecting terminal. And then, acquiring training knowledge field matrixes corresponding to a plurality of training connection terminals based on the knowledge fields corresponding to the training connection terminals.
And obtaining a training common measurement result matrix based on the common measurement result between every two training connection terminals, loading the training knowledge field matrix and the training common measurement result matrix into a probabilistic inference network, and inferring to obtain the probability of each training connection terminal. And adjusting the network parameters of the probabilistic inference network based on the loss between the probability obtained by reasoning each training connection terminal and the labeled probability until a preset condition is met, such as model convergence or debugging times.
As an embodiment, the association degree of the connecting line between each connecting terminal and each adjacent connecting terminal is inferred through an association inference network debugged in advance. For each connection terminal, a set of alternatives is determined, the set of alternatives comprising neighboring connection terminals of the connection terminal, the probability being greater than the neighboring connection terminals of the connection terminal. The probability of the adjacent connection terminal in the close region is greater than that of the connection terminal, which represents that the adjacent connection terminal is more likely to classify information for one object, in order to classify the connection terminal into the feature classification information, the connection terminal may be connected to the adjacent connection terminal having a higher probability than that of the connection terminal, an alternative set is determined for the connection terminal, but the selected adjacent connection terminal and the connection terminal may not be the same classification information, and then consideration of the degree of association of the connection terminal needs to be introduced. And then, loading the alternative set to a correlation inference network debugged in advance, wherein the correlation inference network outputs the degree of correlation between the connecting terminal and each connecting line between the nearest adjacent connecting terminals in the alternative set.
The associative inference network and the probabilistic inference network may have similar architectures, but the probabilistic inference network does not process the entire adjacent data set, the associative inference network processes the sub-adjacent data set constructed by the candidate set, and the associative inference network generates a probability that each adjacent connection terminal in the candidate set represents that the adjacent connection terminal and the connection terminal are the same classification information.
As an implementation manner, the associative inference network may be debugged through an internet of things training data set, where each internet of things training data subset in the internet of things training data set is labeled with classification information. The method comprises the steps of obtaining a knowledge field of each training data subset of the Internet of things, using the knowledge field of each training data subset of the Internet of things as a training connecting terminal, and obtaining a training alternative set for each training connecting terminal, wherein the alternative set comprises adjacent connecting terminals of the training connecting terminals, and the probability of the alternative set is larger than that of the adjacent connecting terminals of the training connecting terminals. Then connecting the training connecting terminal with each adjacent connecting terminal in the alternative set to obtain a training connecting line, determining the association degree of the training connecting line based on the classification information of the training connecting terminal and the classification information of the adjacent connecting terminal, marking the training connecting line, determining the association degree of the connecting line between the training connecting terminal and the adjacent connecting terminal as 1 when the classification information of the training connecting terminal and the adjacent connecting terminal is consistent, and determining the association degree between the training connecting terminal and the adjacent connecting terminal as 0 when the classification information of the training connecting terminal and the adjacent connecting terminal is inconsistent; and loading the alternative set to an associated reasoning network, reasoning to obtain the association degree of each connecting line, and adjusting the network parameters of the associated reasoning network until reaching a preset condition based on the loss between the association degree and the labeled association degree of each training connecting line to obtain the debugged associated reasoning network.
Based on the same principle as the method shown in fig. 1, the embodiment of the present application further provides a data acquisition apparatus 10, as shown in fig. 2, where the apparatus 10 includes:
the analysis module 11 is configured to, when acquiring the internet of things data packet uploaded by the internet of things sensor network, analyze the internet of things data packet to obtain a first internet of things data relationship network, where the first internet of things data relationship network includes a plurality of first data cluster network nodes.
The mining module 12 is configured to obtain a first relationship network description vector of the first internet of things data relationship network and a first data cluster network node description vector of a plurality of first data cluster network nodes in the first internet of things data relationship network. The first data cluster network nodes are network nodes corresponding to data cluster information in the data packet of the Internet of things, and the first Internet of things data relation network is established based on the involvement conditions among the plurality of first data cluster network nodes.
A data significant factor determining module 13, configured to process the first relational network description vector and the plurality of first data cluster network node description vectors, and determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network. Wherein the data significance factor characterizes representative information of the first data cluster network node in the first Internet of things data relationship network.
The screening module 14 is configured to determine, based on the obtained multiple data significance factors, multiple second data cluster network nodes from among the multiple first data cluster network nodes, where data significance factors corresponding to the multiple second data cluster network nodes are greater than data significance factors corresponding to the remaining first data cluster network nodes.
And the storage classification module 15 is used for extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vectors of the second relation data relation network to determine the storage classification information of the internet of things data packet. Wherein the second associative data relationship network is established based on the involvement between the plurality of second data cluster network nodes.
The foregoing embodiment introduces the data acquisition apparatus 10 from the perspective of a virtual module, and the following introduces a data acquisition cloud platform from the perspective of a physical module, as follows:
an embodiment of the present application provides a data acquisition cloud platform, as shown in fig. 3, the data acquisition cloud platform 100 includes: a processor 101 and a memory 103. Wherein the processor 101 is coupled to the memory 103, such as via a bus 102. Optionally, the data acquisition cloud platform 100 may further include a transceiver 104. It should be noted that, in practical applications, the transceiver 104 is not limited to one, and the structure of the data acquisition cloud platform 100 does not constitute a limitation to the embodiments of the present application.
The processor 101 may be a CPU, general purpose processor, GPU, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein. The processor 101 may also be a combination of computing functions, e.g., comprising one or more microprocessors in combination, a DSP and a microprocessor in combination, or the like.
Bus 102 may include a path that conveys information between the aforementioned components. The bus 102 may be a PCI bus or EISA bus, etc. The bus 102 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.
The memory 103 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The memory 103 is used for storing application program codes for executing the scheme of the application, and the execution is controlled by the processor 101. The processor 101 is configured to execute application program code stored in the memory 103 to implement the aspects of any of the method embodiments described above.
The embodiment of the application provides a data acquisition cloud platform, and the data acquisition cloud platform in the embodiment of the application comprises: one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more programs are executed by the processors, the method for data collection based on artificial intelligence internet of things as described above is performed. According to the technical scheme, the data significance factors of the first data cluster network nodes are obtained through the relational network description vectors and the data cluster network node description vectors, richer reference information is covered, the obtained data significance factors can fully reflect the representativeness of the data cluster network nodes in the Internet of things data relational network, the representative data cluster network nodes can be accurately determined in the process of selecting the first data cluster network nodes based on the reference data significance factors, the Internet of things data packets are analyzed and refined based on the obtained data cluster network node description vectors and the relational network description vectors of the representative data cluster network nodes, the accuracy of data classification is improved, the data cluster network nodes with poor representativeness are not included in the analysis category, and the efficiency of the whole storage classification process is effectively improved.
The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a processor, enables the processor to execute the corresponding content in the foregoing method embodiments.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.

Claims (10)

1. The data acquisition method based on the artificial intelligence Internet of things is applied to the data acquisition cloud platform, the data acquisition cloud platform is in communication connection with at least one Internet of things sensing network, the Internet of things sensing network comprises an Internet of things relation database, the Internet of things relation database is used for storing Internet of things data packets, and the method comprises the following steps:
when an internet of things data packet uploaded by the internet of things sensor network is acquired, analyzing the internet of things data packet to acquire a first internet of things data relation network, wherein the first internet of things data relation network comprises a plurality of first data cluster network nodes;
acquiring a first relation network description vector of the first internet of things data relation network and first data cluster network node description vectors of a plurality of first data cluster network nodes in the first internet of things data relation network; the first data cluster network node is a network node corresponding to data cluster information in the data packet of the internet of things, and the first data relation network of the internet of things is established based on the involvement condition among the plurality of first data cluster network nodes;
processing the first relational network description vector and the plurality of first data cluster network node description vectors to determine a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network; wherein the data significance factor characterizes representative information of the first data cluster network node in the first Internet of things data relationship network;
determining a plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the obtained plurality of data significance factors, wherein the data significance factors corresponding to the plurality of second data cluster network nodes are larger than the data significance factors corresponding to the rest of first data cluster network nodes;
extracting a first data cluster network node description vector of the plurality of second data cluster network nodes and a second relation network description vector of a second relation data relation network to determine storage classification information of the data packet of the internet of things; wherein the second associative data relationship network is established based on the involvement between the plurality of second data cluster network nodes.
2. The method of claim 1, wherein after the step of determining a plurality of second data cluster nodes among the plurality of first data cluster nodes based on the obtained plurality of data saliency factors, the method further comprises:
modifying a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relational network description vector to obtain a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors, and determining a data significance factor of each second data cluster network node in the second relational network;
determining a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the obtained plurality of data significant factors, wherein the data significant factors corresponding to the plurality of third data cluster network nodes are larger than the data significant factors corresponding to the rest of the second data cluster network nodes.
3. The method according to claim 2, wherein the extracting the first data cluster node description vector of the plurality of second data cluster nodes and the second relational network description vector of the second relational network to determine the storage classification information of the internet of things data packet comprises:
extracting a first data clustering network node description vector of the second data clustering network nodes, the second relation network description vector, a second data clustering network node description vector of the third data clustering network nodes and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement conditions among the third data clustering network nodes.
4. The method according to claim 3, wherein the extracting the first data cluster network node description vector of the second data cluster network nodes, the second relation network description vector, the second data cluster network node description vector of the third data cluster network nodes, and the third relation network description vector of the third internet of things node to determine the storage classification information of the internet of things data packet comprises:
based on the numerical statistical result of the second data cluster network nodes, performing mean calculation on the first data cluster network node description vectors and the second relation network description vectors of the plurality of second data cluster network nodes to determine data cluster network node mean description vectors corresponding to the plurality of second data cluster network nodes;
connecting the data cluster network node mean description vector with a maximum data cluster network node description vector in a first data cluster network node description vector of the plurality of second data cluster network nodes to obtain the first combined description vector;
splicing second data clustering network node description vectors of the plurality of third data clustering network nodes and the third relation network description vector to obtain a second combined description vector;
connecting the first merging description vector with the second merging description vector, and determining a merging description vector corresponding to the Internet of things data packet;
and extracting the merged description vector, and determining storage classification information of the Internet of things data packet.
5. The method according to claim 1, wherein the data saliency determination module network node selection module classifies the first relational network description vector and the plurality of first data cluster network node description vectors, and the step of determining the data saliency corresponding to each first data cluster network node in the first internet of things data relational network is performed by a pre-debugged storage classification network, which includes a first data saliency determination module, a first network node selection module, and a classification module, and the step of processing the first relational network description vector and the plurality of first data cluster network node description vectors to determine the data saliency corresponding to each first data cluster network node in the first internet of things data relational network specifically includes:
processing the first relational network description vector and the plurality of first data cluster network node description vectors through the first data significant factor determination module, and determining a data significant factor corresponding to each first data cluster network node in the first internet of things data relational network;
the determining a plurality of second data cluster nodes among the plurality of first data cluster nodes based on the obtained plurality of data saliency factors includes:
determining, by the first network node selection module, the plurality of second data cluster network nodes in the plurality of first data cluster network nodes based on the obtained plurality of data saliency factors;
the extracting the first data cluster network node description vectors of the plurality of second data cluster network nodes and the second relation network description vector of the second relation data relation network to determine the storage classification information of the internet of things data packet includes:
and extracting the first data clustering network node description vector and the second relation network description vector of the plurality of second data clustering network nodes through the classification module, and determining the storage classification information of the data packet of the internet of things.
The storage and classification network further includes a first linear transformation module, a second data saliency factor determination module, and a second network node selection module, and after the first network node selection module determines the plurality of second data clustering network nodes among the plurality of first data clustering network nodes based on the obtained plurality of data saliency factors, the method further includes:
modifying, by the first linear transformation module, a first data cluster network node description vector of the plurality of second data cluster network nodes based on the second relationship network description vector, and determining a second data cluster network node description vector of the plurality of second data cluster network nodes;
processing the second relational network description vector and a plurality of second data cluster network node description vectors through the second data significance factor determining module, and determining a data significance factor of each second data cluster network node in the second relational network;
determining, by the second network node selection module, a plurality of third data cluster network nodes in the plurality of second data cluster network nodes based on the plurality of acquired data significance factors, where data significance factors corresponding to the plurality of third data cluster network nodes are greater than data significance factors corresponding to the remaining second data cluster network nodes.
6. The method according to claim 5, wherein the extracting, by the classification module, the first data cluster node description vector and the second relationship network description vector of the plurality of second data cluster nodes to determine the storage classification information of the internet of things data packet comprises:
extracting, by the classification module, a first data cluster network node description vector of the plurality of second data cluster network nodes, the second relation network description vector, a second data cluster network node description vector of the plurality of third data cluster network nodes, and a third relation network description vector of a third internet of things data relation network, and determining storage classification information of the internet of things data packet, wherein the third internet of things data relation network is established based on involvement conditions among the plurality of third data cluster network nodes.
7. The method according to any one of claims 5 to 6, wherein the storage classification network further comprises a first splicing module and a second splicing module, and the determining the storage classification information of the IOT data packet by extracting a first data clustering net node description vector of the plurality of second data clustering net nodes, the second relation net description vector, a second data clustering net node description vector of the plurality of third data clustering net nodes and a third relation net description vector of a third IOT data relation net through the classification module comprises:
splicing a first data cluster network node description vector and a second relation network description vector of the plurality of second data cluster network nodes through the first splicing module to obtain a first combined description vector;
splicing a second data cluster network node description vector and the third relation network description vector of the plurality of third data cluster network nodes through the second splicing module to obtain a second combined description vector;
extracting the first merging description vector and the second merging description vector through the classification module, and determining storage classification information of the data packet of the Internet of things;
the storage classification network further includes a merging module, and the extracting the first merged description vector and the second merged description vector through the classification module to determine the storage classification information of the internet of things data packet includes:
connecting the first merging description vector and the second merging description vector through the merging module, and determining a merging description vector corresponding to the data packet of the internet of things;
and extracting the merged description vector through the classification module, and determining the storage classification information of the data packet of the Internet of things.
8. The method according to any one of claims 5 to 7, wherein the storage classification network is obtained by debugging the following steps:
acquiring training storage classification information of an Internet of things training data packet, training relation network description vectors of a training Internet of things data relation network and training data cluster network node description vectors of a plurality of training data cluster network nodes in the training Internet of things data relation network, wherein the training data cluster network nodes are network nodes corresponding to the training data cluster information, and the training Internet of things data relation network is established based on the involvement condition among the plurality of training data cluster information in the Internet of things training data packet;
extracting the training relation network description vector and the training data clustering network node description vectors of the training data clustering network nodes through the storage classification network, and determining inference storage classification information of the training data packet of the Internet of things;
and debugging the network parameters of the storage classification network until a preset condition is reached based on the loss between the training storage classification information and the reasoning storage classification information to obtain the storage classification network after debugging.
9. A data acquisition system is characterized by comprising a data processing cloud platform and at least one Internet of things sensing network in communication connection with the data processing cloud platform, wherein the Internet of things sensing network comprises an Internet of things relation database which is used for storing Internet of things data packets, the data processing cloud platform comprises a processor and a memory, a computer program is stored in the memory, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is executed.
10. A data acquisition cloud platform comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, performs the method of any one of claims 1 to 8.
CN202211553105.0A 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform Active CN115599873B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211553105.0A CN115599873B (en) 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211553105.0A CN115599873B (en) 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform

Publications (2)

Publication Number Publication Date
CN115599873A true CN115599873A (en) 2023-01-13
CN115599873B CN115599873B (en) 2023-08-29

Family

ID=84851927

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211553105.0A Active CN115599873B (en) 2022-12-06 2022-12-06 Data acquisition method and system based on artificial intelligence Internet of things and cloud platform

Country Status (1)

Country Link
CN (1) CN115599873B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117273765A (en) * 2023-11-21 2023-12-22 广州欧派创意家居设计有限公司 Multistage dealer circulation data processing method and system based on automatic check

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9311386B1 (en) * 2013-04-03 2016-04-12 Narus, Inc. Categorizing network resources and extracting user interests from network activity
WO2018219223A1 (en) * 2017-05-31 2018-12-06 腾讯科技(深圳)有限公司 Data processing method, device and storage medium
CN109033387A (en) * 2018-07-26 2018-12-18 广州大学 A kind of Internet of Things search system, method and storage medium merging multi-source data
CN109347834A (en) * 2018-10-24 2019-02-15 广东工业大学 Detection method, device and the equipment of abnormal data in Internet of Things edge calculations environment
CN109672750A (en) * 2019-01-15 2019-04-23 特斯联(北京)科技有限公司 A kind of security protection data aggregate management system and its management method based on Edge fabrics
CN110348526A (en) * 2019-07-15 2019-10-18 武汉绿色网络信息服务有限责任公司 A kind of device type recognition methods and device based on semi-supervised clustering algorithm
CN110955212A (en) * 2019-12-05 2020-04-03 河南工业大学 Wisdom agricultural information processing system based on thing networking
CN112104684A (en) * 2020-02-12 2020-12-18 云池网络科技(上海)有限公司 Platform management system based on Internet of things
CN113207101A (en) * 2021-04-13 2021-08-03 山东曙光照信息技术有限公司 Information processing method based on 5G city component sensor and Internet of things cloud platform
US20210352134A1 (en) * 2020-01-19 2021-11-11 Bernt Erik Bjontegard Contextual intelligence of every-thing communication platform integrating edge computation, augmented intelligence and distributed ledger data storage nodes and tokens for real world data interactions across internet of things with payment integration
CN113837009A (en) * 2021-08-26 2021-12-24 张大艳 Internet of things data acquisition and analysis system based on artificial intelligence
CN114257980A (en) * 2021-12-06 2022-03-29 安徽南瑞继远电网技术有限公司 Data interaction system and interaction method based on cluster network
CN114547440A (en) * 2022-01-25 2022-05-27 王金山 User portrait mining method based on internet big data and artificial intelligence cloud system

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9311386B1 (en) * 2013-04-03 2016-04-12 Narus, Inc. Categorizing network resources and extracting user interests from network activity
WO2018219223A1 (en) * 2017-05-31 2018-12-06 腾讯科技(深圳)有限公司 Data processing method, device and storage medium
CN109033387A (en) * 2018-07-26 2018-12-18 广州大学 A kind of Internet of Things search system, method and storage medium merging multi-source data
CN109347834A (en) * 2018-10-24 2019-02-15 广东工业大学 Detection method, device and the equipment of abnormal data in Internet of Things edge calculations environment
CN109672750A (en) * 2019-01-15 2019-04-23 特斯联(北京)科技有限公司 A kind of security protection data aggregate management system and its management method based on Edge fabrics
CN110348526A (en) * 2019-07-15 2019-10-18 武汉绿色网络信息服务有限责任公司 A kind of device type recognition methods and device based on semi-supervised clustering algorithm
CN110955212A (en) * 2019-12-05 2020-04-03 河南工业大学 Wisdom agricultural information processing system based on thing networking
US20210352134A1 (en) * 2020-01-19 2021-11-11 Bernt Erik Bjontegard Contextual intelligence of every-thing communication platform integrating edge computation, augmented intelligence and distributed ledger data storage nodes and tokens for real world data interactions across internet of things with payment integration
CN112104684A (en) * 2020-02-12 2020-12-18 云池网络科技(上海)有限公司 Platform management system based on Internet of things
CN113207101A (en) * 2021-04-13 2021-08-03 山东曙光照信息技术有限公司 Information processing method based on 5G city component sensor and Internet of things cloud platform
CN113837009A (en) * 2021-08-26 2021-12-24 张大艳 Internet of things data acquisition and analysis system based on artificial intelligence
CN114257980A (en) * 2021-12-06 2022-03-29 安徽南瑞继远电网技术有限公司 Data interaction system and interaction method based on cluster network
CN114547440A (en) * 2022-01-25 2022-05-27 王金山 User portrait mining method based on internet big data and artificial intelligence cloud system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117273765A (en) * 2023-11-21 2023-12-22 广州欧派创意家居设计有限公司 Multistage dealer circulation data processing method and system based on automatic check
CN117273765B (en) * 2023-11-21 2024-02-06 广州欧派创意家居设计有限公司 Multistage dealer circulation data processing method and system based on automatic check

Also Published As

Publication number Publication date
CN115599873B (en) 2023-08-29

Similar Documents

Publication Publication Date Title
CN109978228B (en) PM2.5 concentration prediction method, device and medium
CN108615071B (en) Model testing method and device
CN112396160A (en) Transaction fraud detection method and system based on graph neural network
CN113378961A (en) Network traffic identification method, device, equipment and computer program product
CN110809768B (en) Data cleansing system and method
CN116596095B (en) Training method and device of carbon emission prediction model based on machine learning
CN115599873A (en) Data acquisition method and system based on artificial intelligence Internet of things and cloud platform
CN111292377B (en) Target detection method, device, computer equipment and storage medium
CN115062779A (en) Event prediction method and device based on dynamic knowledge graph
CN112257332B (en) Simulation model evaluation method and device
CN117131348B (en) Data quality analysis method and system based on differential convolution characteristics
WO2024078112A1 (en) Method for intelligent recognition of ship outfitting items, and computer device
CN114565092A (en) Neural network structure determining method and device
CN112580780A (en) Model training processing method, device, equipment and storage medium
CN109508582A (en) The recognition methods of remote sensing image and device
CN115484112B (en) Payment big data safety protection method, system and cloud platform
CN114860617B (en) Intelligent pressure testing method and system
CN115984671A (en) Model online updating method and device, electronic equipment and readable storage medium
CN112528500A (en) Evaluation method and evaluation equipment for scene graph construction model
CN112819527A (en) User grouping processing method and device
CN115712614B (en) Information processing method and system based on data transmission flow control
CN112580781A (en) Processing method, device and equipment of deep learning model and storage medium
TWI613545B (en) Analyzing method and analyzing system for graphics process
CN113537407B (en) Image data evaluation processing method and device based on machine learning
CN110750661B (en) Method, device, computer equipment and storage medium for searching image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20230807

Address after: 510000 room 6-211, Part 5, 6 and 7, No. 67, Dongpu Second Road, Tianhe District, Guangzhou City, Guangdong Province (office only)

Applicant after: Guangzhou Fengwang Internet Technology Co.,Ltd.

Address before: No. 109, Ankang Street, Bincheng District, Binzhou City, Shandong Province, 256600 - 0085

Applicant before: Binzhou Yuwei Network Technology Co.,Ltd.

GR01 Patent grant
GR01 Patent grant