CN109886825B - Agricultural Internet of things data multi-view projection clustering reconstruction method and system - Google Patents
Agricultural Internet of things data multi-view projection clustering reconstruction method and system Download PDFInfo
- Publication number
- CN109886825B CN109886825B CN201811594725.2A CN201811594725A CN109886825B CN 109886825 B CN109886825 B CN 109886825B CN 201811594725 A CN201811594725 A CN 201811594725A CN 109886825 B CN109886825 B CN 109886825B
- Authority
- CN
- China
- Prior art keywords
- matrix
- data
- plane
- projection matrix
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the invention provides a method and a system for multi-view projection cluster reconstruction of agricultural Internet of things data, wherein the method comprises the following steps: acquiring sensing data sent by an agricultural Internet of things, and generating a sensing matrix according to the sensing data; projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix; clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix; and selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the data of the agricultural Internet of things. According to the method provided by the embodiment of the invention, the execution complexity of the clustering algorithm is reduced through a multi-view projection technology, and the high-efficiency and accurate reconstruction of the multi-dimensional complex agricultural WSN data is realized.
Description
Technical Field
The embodiment of the invention relates to the technical field of Internet of things, in particular to a method and a system for multi-view projection cluster reconstruction of agricultural Internet of things data.
Background
The agricultural internet of things is an important data source for sensing agricultural environment to conduct agricultural production decision management, scientific research and the like, and the precision and the quality of data have important influence on research and decision results. As the faults of software and hardware such as sensors, network links, acquisition nodes and the like are difficult to avoid, the problems of data errors, data loss and the like are caused, and the quality of perception data is further reduced. The agricultural environment monitoring network has the advantages that the operation is complex and severe, the channel condition is complex, the energy is limited, and the like, the fault occurrence probability is improved, and moreover, the frequency and the type of the fault are increased along with the expansion of the network scale. In order to ensure the integrity and quality of agricultural internet of things data and provide support for agricultural production research and the like, an effective data reconstruction method becomes a key for resisting network faults and data loss.
The current commonly used data interpolation methods include linear interpolation, moving average, multiple regression, natural neighbor interpolation, nearest neighbor interpolation, etc. Data reconstruction algorithms based on machine learning, such as K-nearest neighbor (KNN), Delaunay Triangulation (DT) and multi-channel singular spectral analysis (MSSA). These methods are generally only suitable for few missing value scenes, and when the number of missing values is large, the performance of the methods is obviously reduced
The invention provides a method for realizing the safety data transmission by using a compressed sensing theory, which is found by the document retrieval, and the invention patent 'safety data transmission method based on the compressed sensing theory' (application number: 201210392212. X). A source end uses a random projection matrix and a random vector to encrypt, and a destination end uses a submatrix of a left zero matrix of the random projection matrix to decrypt, so that the data to be transmitted can be reconstructed. The method mainly aims at the encryption and decryption problems in a safe scene, and does not relate to the reconstruction problems of missing and abnormal data. The invention patent ' a sensor data transmission method, a system and a medium ' (application number: 201711084602.X) ' provides a low-rate Internet of things platform sensor data transmission method, which reconstructs acquired data according to a preset data frame format to obtain reconstructed data; transmitting the reconstruction data to a server which establishes communication connection with the low-speed Internet of things platform; the method has the main effects of unifying the data transmission formats of the sensors and improving the transmission efficiency of the Internet of things. Kong et al in the article "Data Loss and Reconstruction in Wireless Sensor Networks" propose a space-time improved compressive sensing and Reconstruction method with the same high Data Loss rate, and give out Reconstruction accuracy analysis from the angle of single parameter and multiple parameters. Sun et al, in "Compressed-Sensing Reconstruction Based on Block Sparse Learning in Bearing-conditioning Monitoring," propose a data Block-oriented Sparse Bayesian Learning (BSBL) algorithm, which reconstructs the transform domain of CS Sparse coefficients and further recovers the original signal by using the Block attributes and the signals of the internal structure.
The traditional interpolation method mainly solves the interpolation filling problem of individual lost data of a time sequence, wherein the linear interpolation method is only suitable for solving the linear problem, the interpolation result of nonlinear data is poor, and the lagrange interpolation method and the like improve the interpolation preparation of the nonlinear data, but are still difficult to solve the problems of multi-valued functions and the like. The latest machine learning-based method provides a good technical means for the problem of wireless sensor network data reconstruction, but the current research mostly stays on the problem of reconstruction of single parameter, single dimension and a small amount of missing data. Data reconstruction research on a large number of data missing scenes is few, technologies such as multi-parameter and multi-dimensional correlation analysis are not adopted for reconstruction analysis, and particularly, the problem of data abnormal missing caused by different abnormalities is not processed in a distinguishing mode.
The agricultural internet of things has a complex and severe working environment, the data of the internet of things is lost or abnormal due to faults of sensors, node software and hardware, communication links and the like, and the reconstruction of agricultural WSN data through the associative clustering among the data is a problem to be researched and solved.
Disclosure of Invention
The embodiment of the invention provides a method and a system for multi-view projection clustering reconstruction of agricultural Internet of things data, which are used for solving the problems that in the prior art, the working environment of the agricultural Internet of things is complicated and severe, and faults of sensors, node software and hardware, communication links and the like can cause loss or abnormality of the Internet of things data.
In a first aspect, an embodiment of the present invention provides an agricultural internet of things data multi-view projection clustering reconstruction method, including:
s1, obtaining perception data sent by the agricultural Internet of things, and generating a perception matrix according to the perception data;
s2, projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix;
s3, clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix by a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix;
and S4, selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the agricultural Internet of things data.
In a second aspect, an embodiment of the present invention provides an agricultural internet of things data multi-view projection clustering reconstruction method and system, including:
the sensing matrix generation module is used for acquiring sensing data sent by the agricultural Internet of things and generating a sensing matrix according to the sensing data;
the projection module is used for projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix;
the clustering module is used for clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix;
and the data reconstruction module is used for selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the data of the agricultural Internet of things.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the program to implement the steps of the method for reconstructing multi-view projection clusters of data of an agricultural internet of things as provided in the first aspect.
In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method for multi-view projection cluster reconstruction of data of an agricultural internet of things as provided in the first aspect.
According to the agricultural Internet of things data multi-view projection clustering reconstruction method and system, provided by the embodiment of the invention, the space-time parameter association characteristic of an agricultural environment is considered, the dimensionality reduction processing of data is realized through a multi-view projection technology, the execution complexity of a clustering algorithm is greatly reduced, meanwhile, the correlation among the data is searched by adopting a clustering method, the calculation complexity of data association analysis is simplified, and further, the efficient and accurate reconstruction of multi-dimensional complex agricultural WSN data is realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a multi-view projection clustering reconstruction method for data of an agricultural internet of things according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a spatial plane mapping matrix in the agricultural internet of things data multi-view projection clustering reconstruction method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a multi-view projection clustering reconstruction method for data of an agricultural internet of things according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of a system of agricultural internet of things data multi-view projection clustering reconstruction method provided in an embodiment of the invention
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a method for reconstructing agricultural internet of things data from multi-view projection clustering, provided in an embodiment of the present invention, where the provided method includes:
s1, obtaining perception data sent by the agricultural Internet of things, and generating a perception matrix according to the perception data;
s2, projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix;
s3, clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix;
and S4, selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the agricultural Internet of things data.
Specifically, for an application scenario of WSN environment data reconstruction, perception data can be generally divided into three dimensions, namely a time dimension, a space dimension and a parameter dimension, and processing of high-dimensional data not only reduces the execution efficiency of a clustering algorithm, but also is difficult to quickly find a hyperplane which completely meets requirements for class division. Therefore, the invention introduces a multi-view projection method, firstly projects agricultural WSN data on different parameter dimensions to form corresponding parameter space-time data, and generally adopts a matrix form to represent the data before and after reconstruction. Firstly, determining the size of a sliding window based on sampling frequency and a data period, and intercepting a perception matrix according to the determined sliding window to serve as input data of model training; and the input data (perceptual matrix) is normalized. The parameter matrix em (environment matrix) is defined as:
X(x(i,j,k))n×t×p
wherein, i is the ith node, and j is the jth time point. k represents the kth parameter, n nodes are arranged in the network, each node perceives p parameters, data of t time points are included in the time window processed by the algorithm, and X is an n multiplied by t multiplied by p matrix.
The corresponding parameter dimensions in the agricultural WSN sensing data include but are not limited to ambient temperature, ambient humidity, soil temperature, soil humidity, illumination intensity, soil PH value, wind speed, wind direction, rainfall and the like.
The WSN data has data loss or abnormal data due to software and hardware faults or channel links and other problems, wherein the abnormal data is deleted after being detected by an abnormal detection algorithm, and can also be treated as lost data together. At this time, if a 0-valued entry appears in the EM Matrix, a Data loss Matrix (DMM) is defined to characterize the loss of Data:
the data actually collected by the WSN can be represented as a perception matrix PM＝B.×X。
For agricultural WSN data reconstruction methods, i.e. data matrix P to be obtained from the acquisitionMThe data matrix X' is recovered in such a way that it is as close as possible to the original data matrix X. Particularly, the agricultural internet of things data is space-time data and has various parameters such as air temperature, air humidity, soil temperature, soil humidity, illumination, air pressure, carbon dioxide concentration and the like, so the agricultural internet of things data is high-dimensional space-time data, and data dimensionality reduction, namely attribute reduction, is firstly carried out on the agricultural internet of things data.
Due to the space-time parameter high-dimensional characteristic of agricultural WSN data, the traditional method is difficult to realize high-efficiency analysis and clustering, so the embodiment provides a multi-view projection data dimension reduction method, correlation analysis is carried out on different dimension attributes of the data before clustering, and the dimension attribute with high correlation is selected to carry out multi-view projection to reduce the data dimension, and the specific steps are as follows:
perception matrix PMMapping to a certain plane of time, space and parameters according to a preset rule, and then sensing the matrix PMThe three-dimensional matrix is transformed into a two-dimensional matrix. For example, the matrix after mapping to the spatial plane may be represented as shown in fig. 2, where data of the jth (j e (1, n)) node in PM is represented. The post-mapping matrices corresponding to the parameter plane at time can be expressed as (i e (1, t)) and (k e (1, p)), respectively.
When multi-view projection clustering is carried out, the actual situation of data loss can be solved
To further simplify the computational complexity of the clustering algorithm, one or more of the above-mentioned methods may be used
Performing a reprojection reduction on
For example, the minimum subspace matrix is selected
As a result of the reduction.
The following conditions need to be satisfied simultaneously:
In specific implementation, a preset clustering algorithm is adopted for clustering projection data matrixes of all visual angles, an intra-class mean value is used as a reconstruction value of missing data, a corresponding deviation Var is calculated, and a result with the minimum Var in three dimensions is selected as reconstruction data under a single-point random loss model.
Correspondingly, when the clustering method is used, the algorithm model needs to be trained first, and the preset loss function measures and calculates training sample data according to the missing form of data to obtain the optimal clustering value number.
By the method, the spatial-temporal parameter association characteristics of the agricultural environment are considered, the dimensionality reduction processing of the data is realized through a multi-view projection technology, the execution complexity of a clustering algorithm is greatly reduced, meanwhile, the correlation among the data is searched by the clustering method, the calculation complexity of data association analysis is simplified, and further the efficient and accurate reconstruction of the multi-dimensional complex agricultural WSN data is realized.
On the basis of the above embodiment, the step after the step of projecting the sensing matrix on the time plane, the space plane and the parameter plane to obtain the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix further includes: judging the loss type of the sensing data of the agricultural Internet of things according to the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix; and if the loss type of the perception data of the agricultural internet of things is the single-point random loss type, continuing to execute S3.
After the step of judging the loss type of the sensing data of the agricultural internet of things according to the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix, the method further comprises the following steps:
s21, if the loss type of the perception data of the agricultural Internet of things is a block random loss type, projecting the perception matrix on a plane of the type according to the type of the loss data corresponding to the block random loss to obtain a plurality of block loss projection matrixes.
And S22, clustering the block loss projection matrixes by adopting a preset clustering algorithm to obtain the deviation value of each block loss projection matrix.
S23, selecting the N block lost projection matrixes with the minimum deviation value as reconstruction data, and performing data reconstruction on the perception matrixes to obtain the perception matrixes after data reconstruction.
S24, using the sensing matrix after the data reconstruction as a new sensing matrix, and executing S2.
Wherein, after the step of reconstructing the data of the agricultural internet of things by taking the sensing matrix after the data reconstruction as a new sensing matrix, the method further comprises the following steps: judging the data loss type of the sensing matrix after reconstruction according to a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix generated by the projection of the sensing matrix after reconstruction; and if the reconstructed sensing matrix data missing type is the block random missing type, re-executing the step S21.
Specifically, referring to fig. 3, fig. 3 is a schematic flow chart of a multi-view projection clustering reconstruction method for agricultural internet of things data of the internet of things according to another embodiment of the present invention, a conventional data loss model generally mainly includes a random loss model, but for WSN data acquisition applications, the data loss model mainly includes the following:
single point random loss model, which is the simplest data loss model. The data in the matrix is independently randomly discarded, i.e. the missing data points are randomly distributed in the monitoring matrix PM. Noise and collisions of typical WSNs are the root cause of this pattern.
The block random loss model is expressed as a phenomenon that adjacent partial data are simultaneously lost in a data matrix PM, and is mainly divided into a spatial sequence block loss model, a time sequence block loss model and a parameter sequence block loss model according to different arrangement dimensions of the adjacent data.
The time sequence block loss model is that data of a certain node is frequently lost in a time sequence, and can be expressed as persistent loss and intermittent loss. In an agricultural WSN application scene, an unreliable link is a common phenomenon, and when the link quality is poor, time sequence blocks are easy to lose in perception data.
The space sequence block model is that data of adjacent nodes on a certain time node are lost together. Agricultural WSN network congestion is a major cause of data loss in high-density multi-sensor nodes.
The loss of the parameter sequence blocks shows that a plurality of parameters of a certain node are lost simultaneously, and the hardware fault of the sensor of the agricultural WSN node is the main reason for the loss of the parameter sequence blocks.
In practical application, the loss of the hybrid loss model is generally caused by multiple factors, but the hybrid loss model is complex and is generally decomposed into the first three models for processing during specific analysis.
Correspondingly, in an agricultural internet of things data loss model, the data loss proportion is generally high, and therefore two problems of high-dimensional data and high loss rate need to be considered simultaneously in agricultural WSN data reconstruction.
For common link faults, network blocking faults, sensor hardware faults and the like in the agricultural WSN, time, space and parameter sequence block loss models of the block random loss models are respectively corresponding, so that the following method steps are adopted during data reconstruction:
firstly, judging which kind of sequence block loss model belongs to, and then determining the perception matrix PMAnd performing multi-view projection in corresponding dimensions and obtaining a corresponding projection matrix. E.g. projection onto the time plane if the time series block is lost, obtaining several projection matrices onto the time plane, e.g. if at i e i0,i1]For block loss, i is generated1-i0+1 projection matrices.
And clustering the obtained projection matrix by adopting a K-means + + method, wherein J (-) is adopted as a loss function of the K-means + + method in the embodiment of the invention. Without loss of generality, similar effects can be achieved by adopting other clustering methods in the step.
When the clustering method is used, the algorithm model needs to be trained firstly, and the training sample data is measured and calculated as a loss function according to the missing form of the data, so that the optimal clustering value quantity is obtained. Using the intra-class mean value as a reconstruction value of missing data, calculating a corresponding deviation Var, and selecting N with the minimum Var from a plurality of projection matrixes0The reconstruction data corresponding to the time plane is generally N0Half of the total projection matrix number; adding reconstructed data to the original data matrix PMIn (b) to obtain P’MThen P is’MThe missing part in (a) may be partly converted to a single point loss while part is still a block loss. If the block loss is still the block loss, the data reconstruction step of the block loss is continuously executed, and if the block loss is converted into the single point loss, the reconstruction is carried out in a single point loss mode.
By the method, the data loss and the abnormality caused by various typical fault types have obvious characteristic rules in consideration of sensor faults, node software and hardware faults, communication link faults and the like in the agricultural WSN, and the accuracy and the efficiency of data reconstruction are further improved by adopting different judging modes for different types of loss abnormal data.
On the basis of the above embodiment, the preset clustering algorithm includes, but is not limited to, any one of K-means + +, density clustering, and mean clustering.
The step of clustering by adopting a preset clustering algorithm specifically comprises the following steps:
clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix by adopting J (-) as a loss function of a K-means + + method; wherein:
in the formula, W is a coefficient matrix of a least square method, rho is a penalty term coefficient larger than zero, Z is a training sample matrix, and Y is a testing sample matrix.
The step of obtaining the deviation value of the time plane projection matrix, the deviation value of the space plane projection matrix and the deviation value of the parameter plane projection matrix specifically comprises: taking the clustering value of each matrix as a reconstruction value of missing data and calculating a corresponding deviation value Var; wherein:
in the formula (I), the compound is shown in the specification,
is a second-order Frobenius norm, X is a matrix before reconstruction, and X' is a matrix after reconstruction.
Specifically, the conventional method mostly adopts the correlation of time and space to carry out correlation estimation, and besides the correlation, part of parameters of the WSN in the agricultural scene also have obvious correlation and periodicity among the parameters. Therefore, a new data estimation and reconstruction method is provided in the embodiment to solve the problem of abnormal data loss in an agricultural WSN scene, and the existing algorithm is improved in the aspect of second-order correlation among parameters.
The recovery matrix and the original matrix are as close as possible as an algorithm optimization target, namely, a reconstruction deviation Var is defined as:
wherein the content of the first and second substances,
is a second-order Frobenius norm, and X' are a reconstructed front matrix and a reconstructed rear matrix respectively. Introducing a least square method as a loss function, and selecting l in consideration of the high-dimensional characteristics of agricultural WSN data2,1Norm comparisonGood fusion of l1The sparsity of norm is characterized by l2The norm prevents the characteristics of loss function overfitting, thus defining a loss function J (·):
in the embodiment of the invention, J (-) is used as a loss function of a K-means + + method, wherein W is a coefficient matrix of a least square method, rho is a penalty term coefficient larger than zero, Z is a training sample matrix, and Y is a testing sample matrix. .
In summary, the invention provides a data association reconstruction method based on clustering, which well solves the problem of local overfitting of a clustering method while ensuring the matrix sparsity by taking a modified F (2, 1) norm as a loss function, and improves the reconstruction accuracy of high-deletion-rate nonlinear data. Meanwhile, a multi-view projection method is provided to realize the dimension reduction of data, simplify the calculation complexity of data association analysis and further realize the efficient and accurate reconstruction of the multi-dimensional complex agricultural WSN data.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an agricultural internet of things data clustering reconstruction system provided in an embodiment of the present invention, where the provided system includes: a perception matrix generation module 41, a projection module 42, a clustering module 43 and a data reconstruction module 44.
The sensing matrix generation module 41 is configured to obtain sensing data sent by an agricultural internet of things and generate a sensing matrix according to the sensing data;
the projection module 42 is configured to project the sensing matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix;
the clustering module 43 is configured to cluster the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix by using a preset clustering algorithm, and obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix, respectively;
the data reconstruction module 44 is configured to select data in the projection matrix with the minimum deviation value as reconstruction data to perform agricultural internet of things data reconstruction.
Wherein the system further comprises: the judging module is used for judging the loss type of the sensing data of the agricultural Internet of things according to the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix; and if the loss type of the sensing data of the agricultural Internet of things is a single-point random loss type, continuously executing the clustering module.
Specifically, for an application scenario of WSN environment data reconstruction, perception data can be generally divided into three dimensions, namely a time dimension, a space dimension and a parameter dimension, and processing of high-dimensional data not only reduces the execution efficiency of a clustering algorithm, but also is difficult to quickly find a hyperplane which completely meets requirements for class division. Therefore, the invention introduces a multi-view projection method, firstly projects agricultural WSN data on different parameter dimensions to form corresponding parameter space-time data, and generally adopts a matrix form to represent the data before and after reconstruction. Firstly, determining the size of a sliding window based on sampling frequency and a data period, and intercepting a perception matrix according to the determined sliding window to serve as input data of model training; and the input data (perceptual matrix) is normalized. The parameter matrix em (environment matrix) is defined as:
X(x(i,j,k))n×t×p
wherein, i is the ith node, and j is the jth time point. k represents the kth parameter, n nodes are arranged in the network, each node perceives p parameters, data of t time points are included in the time window processed by the algorithm, and X is an n multiplied by t multiplied by p matrix.
The corresponding parameter dimensions in the agricultural WSN sensing data include but are not limited to ambient temperature, ambient humidity, soil temperature, soil humidity, illumination intensity, soil PH value, wind speed, wind direction, rainfall and the like.
The WSN data has data loss or abnormal data due to software and hardware faults or channel links and other problems, wherein the abnormal data is deleted after being detected by an abnormal detection algorithm, and can also be treated as lost data together. At this time, if a 0-valued entry appears in the EM Matrix, a Data loss Matrix (DMM) is defined to characterize the loss of Data:
the data actually collected by the WSN can be represented as a perception matrix PM＝B.×X。
For agricultural WSN data reconstruction methods, i.e. data matrix P to be obtained from the acquisitionMThe data matrix X' is recovered in such a way that it is as close as possible to the original data matrix X. Particularly, the agricultural internet of things data is space-time data and has various parameters such as air temperature, air humidity, soil temperature, soil humidity, illumination, air pressure, carbon dioxide concentration and the like, so the agricultural internet of things data is high-dimensional space-time data, and data dimensionality reduction, namely attribute reduction, is firstly carried out on the agricultural internet of things data.
Due to the space-time parameter high-dimensional characteristic of agricultural WSN data, the traditional method is difficult to realize high-efficiency analysis and clustering, so the embodiment provides a multi-view projection data dimension reduction method, correlation analysis is carried out on different dimension attributes of the data before clustering, and the dimension attribute with high correlation is selected to carry out multi-view projection to reduce the data dimension, and the specific steps are as follows:
perception matrix PMMapping to a certain plane of time, space and parameters according to a preset rule, and then sensing the matrix PMThe three-dimensional matrix is transformed into a two-dimensional matrix. For example, the matrix after mapping to the spatial plane may be represented as shown in fig. 2, where data of the jth (j e (1, n)) node in PM is represented. The post-mapping matrices corresponding to the parameter plane at time can be expressed as (i e (1, t)) and (k e (1, p)), respectively.
When multi-view projection clustering is carried out, the actual situation of data loss can be solved
To further simplify the computational complexity of the clustering algorithm,need to be aligned with
Performing a reprojection reduction on
For example, the minimum subspace matrix is selected
As a result of the reduction.
The following conditions need to be satisfied simultaneously:
In specific implementation, a K-means + + method is adopted for clustering projection data matrixes of all visual angles, an intra-class mean value is used as a reconstruction value of missing data, a corresponding deviation Var is calculated, and the result with the minimum Var in three dimensions is selected as reconstruction data under a single-point random loss model.
Correspondingly, when the clustering method is used, the algorithm model needs to be trained first, and the preset loss function measures and calculates training sample data according to the missing form of data to obtain the optimal clustering value number.
By the system, the spatial-temporal parameter association characteristics of the agricultural environment are considered, the dimensionality reduction processing of the data is realized through a multi-view projection technology, the execution complexity of a clustering algorithm is greatly reduced, meanwhile, the correlation among the data is searched by adopting a clustering method, the calculation complexity of data association analysis is simplified, and further the efficient and accurate reconstruction of the multi-dimensional complex agricultural WSN data is realized.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 5, the electronic device includes: a processor (processor)501, a communication Interface (Communications Interface)502, a memory (memory)503, and a bus 504, wherein the processor 501, the communication Interface 502, and the memory 503 are configured to communicate with each other via the bus 504. The processor 501 may call logic instructions in the memory 503 to perform methods including, for example: acquiring sensing data sent by an agricultural Internet of things, and generating a sensing matrix according to the sensing data; projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix; clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix; and selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the data of the agricultural Internet of things.
An embodiment of the present invention discloses a computer program product, which includes a computer program stored on a non-transitory computer readable storage medium, where the computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the method provided by the above method embodiments, for example, the method includes: acquiring sensing data sent by an agricultural Internet of things, and generating a sensing matrix according to the sensing data; projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix; clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix; and selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the data of the agricultural Internet of things.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the methods provided by the above method embodiments, for example, including: acquiring sensing data sent by an agricultural Internet of things, and generating a sensing matrix according to the sensing data; projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix; clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm to respectively obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix; and selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the data of the agricultural Internet of things.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. The agricultural Internet of things data multi-view projection clustering reconstruction method is characterized by comprising the following steps:
s1, obtaining perception data sent by the agricultural Internet of things, and generating a perception matrix according to the perception data;
the generating a perception matrix according to the perception data comprises: projecting the perception data on different parameter dimensions based on introducing a multi-view projection method to obtain parameter space-time data, wherein the parameter space-time data is expressed in a matrix form;
determining the size of a sliding window based on the sampling frequency and the data period, and intercepting the parameter space-time data by the sliding window to generate the perception matrix;
S2projecting the perception matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix
Spatial plane projection matrix
Sum-parameter planar projection matrix
The perceptual matrix is represented as: x (X (i, j, k))n×t×p，
And
representing the matrix after the perception matrix is mapped to a space plane, a time plane and a parameter plane;
wherein, i is the ith node, and j is the jth time point. k represents the kth parameter, n nodes are arranged in the network, each node senses p parameters, data of t time points are included in the time window processed by the algorithm, and X is an n multiplied by t multiplied by p matrix;
s3, clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm, and respectively obtaining an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix;
and S4, selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the agricultural Internet of things data.
2. The method of claim 1, wherein the step after projecting the perceptual matrix in a temporal plane, a spatial plane, and a parametric plane to obtain a temporal plane projection matrix, a spatial plane projection matrix, and a parametric plane projection matrix, further comprises:
judging the loss type of the sensing data of the agricultural Internet of things according to the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix;
and if the loss type of the perception data of the agricultural internet of things is the single-point random loss type, continuing to execute S3.
3. The method according to claim 2, wherein after the step of determining the type of loss of the perception data of the agricultural internet of things according to the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix, the method further comprises:
s21, if the loss type of the perception data of the agricultural Internet of things is a block random loss type, performing multi-view projection on the perception matrix in the dimension corresponding to the lost data to obtain a plurality of block lost projection matrixes;
s22, clustering the block loss projection matrixes by adopting a preset clustering algorithm to obtain a deviation value of each block loss projection matrix;
s23, selecting the N block lost projection matrixes with the minimum deviation value as reconstruction data, and performing data reconstruction on the sensing matrixes to obtain sensing matrixes after data reconstruction;
s24, using the sensing matrix after the data reconstruction as a new sensing matrix, and executing S2.
4. The method according to claim 3, wherein after the step of reconstructing the data of the agricultural internet of things by using the sensing matrix after the data reconstruction as a new sensing matrix, the method further comprises:
judging the data loss type of the reconstructed perception matrix according to a time plane projection matrix, a space plane projection matrix and a parameter plane projection matrix generated by the projection of the reconstructed perception matrix;
and if the reconstructed sensing matrix data missing type is a block random missing type, re-executing the step S21.
5. The method of claim 4, wherein the predetermined clustering algorithm includes but is not limited to any one of K-means + +, density clustering, and mean clustering.
6. The method according to claim 5, wherein the step of clustering with a preset clustering algorithm specifically comprises:
clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix by adopting J (-) as a loss function of a K-means + + method;
wherein the content of the first and second substances,
in the formula, W is a coefficient matrix of a least square method, rho is a penalty term coefficient larger than zero, Z is a training sample matrix, and Y is a testing sample matrix.
7. The method of claim 1, wherein the step of obtaining the bias values of the temporal planar projection matrix, the spatial planar projection matrix and the parametric planar projection matrix comprises:
the matrix after reconstruction and the matrix before reconstruction are closest to each other to be taken as an algorithm optimization target, a reconstruction deviation value Var is defined,
in the formula (I), the compound is shown in the specification,
is a second-order Frobenius norm, X is a matrix before reconstruction, and X' is a matrix after reconstruction.
8. The agricultural Internet of things data multi-view projection clustering reconstruction method system is characterized by comprising the following steps:
the sensing matrix generation module is used for acquiring sensing data sent by the agricultural Internet of things and generating a sensing matrix according to the sensing data;
the generating a perception matrix according to the perception data comprises: projecting the perception data on different parameter dimensions based on introducing a multi-view projection method to obtain parameter space-time data, wherein the parameter space-time data is expressed in a matrix form;
determining the size of a sliding window based on the sampling frequency and the data period, and intercepting the parameter space-time data by the sliding window to generate the perception matrix;
a projection module for projecting the sensing matrix on a time plane, a space plane and a parameter plane to obtain a time plane projection matrix
Spatial plane projection matrix
Sum-parameter planar projection matrix
The perceptual matrix is represented as: x (X (i, j, k))n×t×p，
And
representing the matrix after the perception matrix is mapped to a space plane, a time plane and a parameter plane;
wherein, i is the ith node, and j is the jth time point. k represents the kth parameter, n nodes are arranged in the network, each node senses p parameters, data of t time points are included in the time window processed by the algorithm, and X is an n multiplied by t multiplied by p matrix;
the clustering module is used for clustering the time plane projection matrix, the space plane projection matrix and the parameter plane projection matrix through a preset clustering algorithm respectively to obtain an offset value of the time plane projection matrix, an offset value of the space plane projection matrix and an offset value of the parameter plane projection matrix respectively;
and the data reconstruction module is used for selecting the data in the projection matrix with the minimum deviation value as reconstruction data to reconstruct the data of the agricultural Internet of things.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for multi-view projection cluster reconstruction of data of the agricultural internet of things according to any one of claims 1 to 7.
10. A non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for multi-view projection cluster reconstruction of data of the agricultural internet of things according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811594725.2A CN109886825B (en) | 2018-12-25 | 2018-12-25 | Agricultural Internet of things data multi-view projection clustering reconstruction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811594725.2A CN109886825B (en) | 2018-12-25 | 2018-12-25 | Agricultural Internet of things data multi-view projection clustering reconstruction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109886825A CN109886825A (en) | 2019-06-14 |
| CN109886825B true CN109886825B (en) | 2021-06-29 |
Family
ID=66925165
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811594725.2A Active CN109886825B (en) | 2018-12-25 | 2018-12-25 | Agricultural Internet of things data multi-view projection clustering reconstruction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109886825B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990380B (en) * | 2021-05-11 | 2021-08-03 | 物鼎安全科技(武汉)有限公司 | Filling method and system for missing data of Internet of things |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400143A (en) * | 2013-07-12 | 2013-11-20 | 中国科学院自动化研究所 | Data subspace clustering method based on multiple view angles |
| CN105718950A (en) * | 2016-01-22 | 2016-06-29 | 天津中科智能识别产业技术研究院有限公司 | Semi-supervised multi-visual angle clustering method based on structure constraint |
| CN107895202A (en) * | 2017-10-24 | 2018-04-10 | 天津大学 | Short-term wind speed forecasting method based on various visual angles wind speed mode excavation |
| CN108289285A (en) * | 2018-01-12 | 2018-07-17 | 上海海事大学 | A kind of ocean wireless sensor network is lost data and is restored and reconstructing method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650747B (en) * | 2016-11-16 | 2019-08-02 | 佛山科学技术学院 | A kind of compressed sensing based method for extracting characteristics of three-dimensional model |
| CN109041002B (en) * | 2018-08-22 | 2020-05-26 | 中国农业科学院农业信息研究所 | Intelligent agricultural Internet of things signal compression method |
-
2018
- 2018-12-25 CN CN201811594725.2A patent/CN109886825B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400143A (en) * | 2013-07-12 | 2013-11-20 | 中国科学院自动化研究所 | Data subspace clustering method based on multiple view angles |
| CN105718950A (en) * | 2016-01-22 | 2016-06-29 | 天津中科智能识别产业技术研究院有限公司 | Semi-supervised multi-visual angle clustering method based on structure constraint |
| CN107895202A (en) * | 2017-10-24 | 2018-04-10 | 天津大学 | Short-term wind speed forecasting method based on various visual angles wind speed mode excavation |
| CN108289285A (en) * | 2018-01-12 | 2018-07-17 | 上海海事大学 | A kind of ocean wireless sensor network is lost data and is restored and reconstructing method |
Non-Patent Citations (1)
| Title |
|---|
| A spatial-temporal imputation technique for classification with missing data in a wireless sensor network;YuanYuan Li， Lynne E. Parker;《2008 IEEE/RSJ International Conference on Intelligent Robots and Systems》;20080926;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109886825A (en) | 2019-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102573252B1 (en) | Neural network-based quantum error correction decoding method and device, chip | |
| Zhen | Using big data fuzzy K-means clustering and information fusion algorithm in English teaching ability evaluation | |
| Zhang et al. | Velc: A new variational autoencoder based model for time series anomaly detection | |
| CN110087207B (en) | Method for reconstructing missing data of wireless sensor network | |
| CN113992718B (en) | Method and system for detecting abnormal data of group sensor based on dynamic width chart neural network | |
| CN107786959A (en) | The compressed data collection method based on adaptive measuring in wireless sensor network | |
| Xie et al. | Accurate recovery of missing network measurement data with localized tensor completion | |
| Nizam et al. | Real-time deep anomaly detection framework for multivariate time-series data in industrial iot | |
| CN108924148A (en) | A kind of source signal collaborative compression perception data restoration methods | |
| CN115834433B (en) | Data processing method and system based on Internet of things technology | |
| Tasfi et al. | Deep neural networks with confidence sampling for electrical anomaly detection | |
| CN116455941B (en) | Indoor environment multi-source data transmission method and system based on Internet of things | |
| CN116684878B (en) | 5G information transmission data safety monitoring system | |
| Liu et al. | RB-Net: Training highly accurate and efficient binary neural networks with reshaped point-wise convolution and balanced activation | |
| CN115878603A (en) | Water quality missing data interpolation algorithm based on K nearest neighbor algorithm and GAN network | |
| CN109886825B (en) | Agricultural Internet of things data multi-view projection clustering reconstruction method and system | |
| Cheng et al. | Online power system event detection via bidirectional generative adversarial networks | |
| Li et al. | Robust Low-Rank Tensor Completion Based on Tensor Ring Rank via $\ell _ {p,\epsilon} $-Norm | |
| CN102355752B (en) | Wireless sensor network compressed sensing measurement matrix based on expander graph and reconfiguring method | |
| Sekar et al. | Compressed tensor completion: A robust technique for fast and efficient data reconstruction in wireless sensor networks | |
| CN115879505A (en) | Self-adaptive correlation perception unsupervised deep learning anomaly detection method | |
| Jiang et al. | A graph-based approach for missing sensor data imputation | |
| CN113225089B (en) | Industrial robot multichannel monitoring data compression method and system | |
| CN104459695B (en) | Sparsity microwave imaging method based on compression phase restoring | |
| Gkillas et al. | Missing data imputation for multivariate time series in industrial iot: A federated learning approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |