CN113298265A - Heterogeneous sensor potential correlation learning method based on deep learning - Google Patents
Heterogeneous sensor potential correlation learning method based on deep learning Download PDFInfo
- Publication number
- CN113298265A CN113298265A CN202110561563.8A CN202110561563A CN113298265A CN 113298265 A CN113298265 A CN 113298265A CN 202110561563 A CN202110561563 A CN 202110561563A CN 113298265 A CN113298265 A CN 113298265A
- Authority
- CN
- China
- Prior art keywords
- data
- sensor
- learning
- correlation
- sensors
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000013135 deep learning Methods 0.000 title claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000008447 perception Effects 0.000 claims description 13
- 238000013136 deep learning model Methods 0.000 claims description 10
- 230000001364 causal effect Effects 0.000 claims description 3
- 230000005484 gravity Effects 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 238000010219 correlation analysis Methods 0.000 abstract description 10
- 238000005070 sampling Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004587 chromatography analysis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
- Telephone Function (AREA)
Abstract
本发明涉及一种基于深度学习的多种异构传感器潜在相关性分析方法。该方法首先需要从智能设备中采集所需的传感器数据，这些传感器的数据需要使用相应的采样率来进行采集。将采集的传感器数据保存到指定的文件中；其次为了实现分析多种传感器之间潜在的相关性，需要将一个传感器的数据作为参考，然后利用该方法来分析其他的传感器的数据对应于该传感器的数据之间的影响的占比情况；最后，通过分析多种传感器来作为不同的参考对象来得出多种传感器之间相互的关联关系，这样就得出了多种异构传感器之间潜在的相关性表示。
The invention relates to a potential correlation analysis method of multiple heterogeneous sensors based on deep learning. The method first needs to collect the required sensor data from the smart device, and the data of these sensors needs to be collected using the corresponding sampling rate. Save the collected sensor data to a specified file; secondly, in order to analyze the potential correlation between multiple sensors, it is necessary to use the data of one sensor as a reference, and then use this method to analyze the data of other sensors corresponding to the sensor Finally, by analyzing multiple sensors as different reference objects, the correlation between multiple sensors is obtained, so that the potential between multiple heterogeneous sensors is obtained. Relevance representation.
Description
Technical Field
The invention belongs to the technical field of data processing, relates to a deep learning-based heterogeneous sensor potential correlation learning method, and particularly relates to the field of data acquisition and analysis and the field of data correlation of a sensor of a smart phone used by a user.
Background
In recent years, thanks to the development of technologies such as intelligent sensing and pervasive computing, a cooperative sensing technology widely applied to network convergence becomes a research hotspot. With the rapid development of embedded devices, wireless sensor networks, internet of things and intelligent mobile terminals, a universal intelligent sensing system with integrated sensing, computing and communication capabilities is gradually integrated into the daily life of the society, and the capability of intelligently sensing and acquiring data is remarkably enhanced.
The perception capability obtained by using a single perception device is limited, and in recent years, popularization of various intelligent devices, particularly smart phones, enables common people to share various information such as geographic positions, surrounding environments and the like in real time. The performance of the processor is continuously enhanced, and the smart phone is greatly innovated. Increasingly complex sensors such as sensors capable of measuring acceleration, gyroscope sensors, gravity sensors and direction sensors become standard configurations of smart phones. The sensors enable each individual carrying a mobile phone to become a terminal with strong sensing capability, so that a great number of individuals carrying intelligent terminals can form a coordinator sensing network to observe, track and the like specific targets. Learning of the heterogeneous sensor potential relevance is required.
The document "Rajesh G, Charturedi A. correlation analysis and statistical characterization of heterologous sensor data in environmental sensor Networks [ J ]. Computer Networks, 2019" proposes four classical and robust correlation coefficient measurement methods. The paper utilizes the correlation between sensor modalities to complete the subsequent sensor data interpolation and new modality data prediction, and defines and expresses the correlation between modalities by measuring the correlation coefficient between different multi-modality variables.
The document "Jiang M, Gao X, An H, et al.Reconstructing complex network for chromatography of the time-varying evaluation analysis viewer of multi-varying time series [ J ]. Scientific Reports,2017,7 (1): 10486-. The method has the potential of analyzing the multivariate time sequence and provides important information for investors and decision makers;
the document "Gupta S, Buriro a, Crispo b. driverauth: a Risk-based Multi-modal Biometric-based Driver Authentication Scheme [ J ] Computers & Security, 2019' through learning and utilizing the correlation among a plurality of heterogeneous sensors carried in smart phone equipment, a novel Multi-modal Biometric Authentication solution based on risks is realized, and the aim of ensuring the certification service of a network car booking Driver to be safer and more reliable is achieved.
The above documents briefly introduce the existing correlation analysis method and its main application scenario, and the existing correlation analysis method mainly represents the correlation by calculating the corresponding correlation coefficient. But for heterogeneous sensor devices, the data it produces is heterogeneous time series data. For the correlation analysis between the heterogeneous time series data in a large scale, the existing correlation analysis method based on correlation coefficient calculation cannot meet the requirement.
Disclosure of Invention
Technical problem to be solved
In order to solve the problem that the traditional correlation analysis method based on correlation coefficient calculation cannot effectively perform correlation analysis on the perception data of the heterogeneous sensor, the invention provides a heterogeneous sensor potential correlation learning method based on deep learning. The method is used for carrying out potential correlation analysis on a large amount of heterogeneous time sequence data generated by various sensors, and a real-time correlation analysis task is realized by utilizing a deep learning technology. The invention also optimizes the traditional time convolution network model and creatively realizes the task of analyzing the correlation of the time sequence data.
Technical scheme
A heterogeneous sensor potential correlation learning method based on deep learning is characterized by comprising the following steps:
step 1: acquiring perception data of a terminal user by using a sensor;
step 2: storing and preprocessing the acquired sensing data of the sensor;
and step 3: executing a data alignment task on the sensor data file of the acquired data;
and 4, step 4: taking the sensor data after executing the data alignment task as input data of a deep learning model, wherein the output data of the deep learning model is data of any sensor; after learning, the deep learning model returns a two-dimensional matrix to represent the correlation between various sensors which are input currently and output sensors;
and 5: learning all the sensor data as output data of the model, and finally summarizing the corresponding relations among all the sensors to generate a correlation two-dimensional matrix among all the sensors;
step 6: a two-dimensional matrix of all sensor correlations is visualized.
Preferably: the terminal in the step 1 is a smart phone.
Preferably: the sensor in the step 1 is a gyroscope sensor, a gravity sensor or a direction sensor.
Preferably: the file type of the stored sensor data in the step 2 is a CSV file.
Preferably: the data alignment task in step 3 is specifically: selecting the sensor S with the longest data lengthkThen, S iskAnd performing an alignment task on the data of the other sensors as a reference element of the data, selecting the data which is collected by the sensor for the last time to be supplemented when the alignment task is executed, and finally, all the sensors have data with relevant lengths after the execution of the data-optimal task is finished.
Preferably: the deep learning model in the step 4 comprises causal convolution, dilation convolution, residual connection and attention mechanism.
Preferably: the correlation in step 4 is specifically as follows: the decimal between 0 and 1 is used for representation, 0 represents the minimum correlation, and 1 represents the highest correlation.
Preferably: step 6 is visualized using thermodynamic diagrams.
Advantageous effects
The heterogeneous sensor correlation learning method based on deep learning can be used for completing the task of analyzing the correlation between heterogeneous sensors. For a large amount of time sequence data generated by heterogeneous sensors, the method can learn the correlation among the data through an improved time convolution network model, and the improved time convolution network model can express the correlation in a digital form through an attention mechanism of the improved time convolution network model, and finally, potential correlation relations among the heterogeneous sensors can be visually observed through a data visualization technology. The perception capability of the heterogeneous sensors can be fully exerted in intelligent perception and pervasive computing tasks by using the intuitive correlation results, the perception tasks are completed in a more efficient collaborative perception mode, cost reduction and efficiency improvement of the perception tasks are realized, and a plurality of complex user behavior recognition, user context perception and other high-level perception tasks are completed in a more energy-saving mode.
Drawings
FIG. 1 is a flowchart of a deep learning-based method for learning potential correlations of heterogeneous sensors according to an embodiment of the present invention
FIG. 2 is a flow chart of the execution of data alignment tasks during data pre-processing phase in an embodiment of the present invention
FIG. 3 is a diagram of a deep learning-based relevance latent learning neural network model structure constructed in an embodiment of the present invention
Detailed Description
The invention will now be further described with reference to the following examples and drawings:
the method comprises the steps that firstly, sensing data of all sensors are acquired from intelligent equipment carrying a plurality of heterogeneous sensors, and specifically, the same sampling frequency is used for acquiring all acquired data of the heterogeneous sensors; the neural network model is then used to learn the correlation between these sensors, and in particular the inputs and outputs of the model, here the corresponding sensor data, need to be specified. Meanwhile, in order to train the neural network model better, corresponding data is divided into two parts, namely training and testing. Secondly, in order to learn the correlation between the heterogeneous sensors, the influence relationship of the input data relative to the output data needs to be extracted in the model. Therefore, the part is specifically set in the model to learn the correlations, i.e., a matrix is used to hold the correlations. In order to represent the correlation between heterogeneous sensors, a decimal between 0 and 1 is used for representation, 0 represents the minimum correlation, and 1 represents the highest correlation. Next, in order to learn the potential correlation between a plurality of heterogeneous sensors, it is necessary to use all the heterogeneous sensors as the output of the model, and to sequentially input the data of the other sensors, and then to perform model learning and recording a plurality of times. Finally, the learned correlation coefficient data among the various heterogeneous sensor data are sorted, and finally the data are expressed as a two-dimensional matrix. The two-dimensional matrix records the correlations between all heterogeneous sensors and other heterogeneous sensors. Finally, in order to better show the potential correlation among the sensors, the invention also adopts a visualization strategy to express the potential correlation among the heterogeneous sensors learned by the neural network.
In order to realize the task, the invention adopts the following technical scheme: the heterogeneous sensor potential correlation learning method based on deep learning comprises the following steps:
step one, a user uses a portable smart phone to carry out daily activities, and meanwhile, a sensor mounted on the smart phone starts to collect data sensed currently.
And step two, sending the acquired sensing data of the sensor to a server side for storage, wherein the acquired sensor data file type is a CSV file. And after all the files are uploaded, the data acquisition task of the smart phone end is finished.
And thirdly, carrying out data preprocessing operation on the acquired sensor data file to reduce the influence of invalid data. And deleting the sensor files without the acquired data by data preprocessing, and only leaving the sensor data files with the acquired data.
Step four, executing data alignment task on the sensor data file of the acquired data, and executing data alignment taskThe execution flow chart of (2) is as shown in fig. 2. Selecting the sensor S with the longest data lengthk. Then the S iskAs a reference element of the data, the data of the remaining sensors are subjected to an alignment task. And when the alignment task is executed, the data acquired by the sensor at the last time is selected for completion, and finally all the sensors have data with relevant length after the execution of the data completion task is finished.
And step five, taking the sensor data after the data alignment task is executed as input data of the deep learning model, wherein the output data of the deep learning model is data of any sensor. The structure diagram of the deep learning based neural network model is shown in fig. 3, and the model comprises four components: causal convolution, dilation convolution, residual concatenation and attention mechanism. Consists of two layers: a time convolutional network layer, an attention layer. After the input and output data of the model are specified, a model learning task is started, and after the model is learned, a correlation matrix is returned to represent the potential correlation relationship existing between the currently input sensor and the currently specified output sensor.
And step six, selecting other sensors as output data of the model to perform a task of learning the potential correlation between the sensors and the input sensors, and finally performing potential correlation learning by using all the sensors as the output data of the model. And summarizing the learned correlation matrix, respectively displaying the potential correlation between each sensor relative to other sensors, and generating a correlation two-dimensional matrix among all the sensors.
Finally, a two-dimensional matrix visualization thermodynamic diagram representing all sensor correlations is output as a final result.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110561563.8A CN113298265B (en) | 2021-05-22 | 2021-05-22 | Heterogeneous sensor potential correlation learning method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110561563.8A CN113298265B (en) | 2021-05-22 | 2021-05-22 | Heterogeneous sensor potential correlation learning method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298265A true CN113298265A (en) | 2021-08-24 |
| CN113298265B CN113298265B (en) | 2024-01-09 |
Family
ID=77324026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110561563.8A Active CN113298265B (en) | 2021-05-22 | 2021-05-22 | Heterogeneous sensor potential correlation learning method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298265B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220407769A1 (en) * | 2021-06-16 | 2022-12-22 | Ironwood Cyber Inc. | Control system anomaly detection using neural network consensus |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103018181A (en) * | 2012-12-14 | 2013-04-03 | 江苏大学 | Soft measurement method based on correlation analysis and ELM neural network |
| CN105760507A (en) * | 2016-02-23 | 2016-07-13 | 复旦大学 | Cross-modal subject correlation modeling method based on deep learning |
| CN107038506A (en) * | 2017-05-09 | 2017-08-11 | 华东师范大学 | A kind of factory's intelligent early-warning method |
| US20190228272A1 (en) * | 2018-01-23 | 2019-07-25 | Intelligent Fusion Technology, Inc | Methods, systems and media for joint manifold learning based heterogenous sensor data fusion |
| CN110209822A (en) * | 2019-06-11 | 2019-09-06 | 中译语通科技股份有限公司 | Sphere of learning data dependence prediction technique based on deep learning, computer |
| US20190293462A1 (en) * | 2018-03-23 | 2019-09-26 | Industry-Academic Cooperation Foundation, Dankook University | Apparatus and method for processing multi-type sensor signal on the basis of multi-modal deep learning |
| CN112116019A (en) * | 2020-09-27 | 2020-12-22 | 四川大学 | A Multi-sensor Vine Copula Heterogeneous Information Decision Fusion Method |
-
2021
- 2021-05-22 CN CN202110561563.8A patent/CN113298265B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103018181A (en) * | 2012-12-14 | 2013-04-03 | 江苏大学 | Soft measurement method based on correlation analysis and ELM neural network |
| CN105760507A (en) * | 2016-02-23 | 2016-07-13 | 复旦大学 | Cross-modal subject correlation modeling method based on deep learning |
| CN107038506A (en) * | 2017-05-09 | 2017-08-11 | 华东师范大学 | A kind of factory's intelligent early-warning method |
| US20190228272A1 (en) * | 2018-01-23 | 2019-07-25 | Intelligent Fusion Technology, Inc | Methods, systems and media for joint manifold learning based heterogenous sensor data fusion |
| US20190293462A1 (en) * | 2018-03-23 | 2019-09-26 | Industry-Academic Cooperation Foundation, Dankook University | Apparatus and method for processing multi-type sensor signal on the basis of multi-modal deep learning |
| CN110209822A (en) * | 2019-06-11 | 2019-09-06 | 中译语通科技股份有限公司 | Sphere of learning data dependence prediction technique based on deep learning, computer |
| CN112116019A (en) * | 2020-09-27 | 2020-12-22 | 四川大学 | A Multi-sensor Vine Copula Heterogeneous Information Decision Fusion Method |
Non-Patent Citations (2)
| Title |
|---|
| G. RAJESH等: "Correlation analysis and statistical characterization of heterogeneous sensor data in environmental sensor networks", COMPUTER NETWORKS * |
| 丁小欧;于晟健;王沐贤;王宏志;高宏;杨东华;: "基于相关性分析的工业时序数据异常检测", 软件学报, no. 03 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220407769A1 (en) * | 2021-06-16 | 2022-12-22 | Ironwood Cyber Inc. | Control system anomaly detection using neural network consensus |
| US11546205B1 (en) * | 2021-06-16 | 2023-01-03 | Ironwood Cyber Inc. | Control system anomaly detection using neural network consensus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298265B (en) | 2024-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | Simulating land-use dynamics under planning policies by integrating artificial immune systems with cellular automata | |
| CN112509690B (en) | Method, apparatus, device and storage medium for controlling quality | |
| CN109492772B (en) | Method and device for generating information | |
| CN112784778B (en) | Method, apparatus, device and medium for generating model and identifying age and sex | |
| CN110135497B (en) | Model training method, and method and device for estimating strength of facial action unit | |
| WO2024041479A1 (en) | Data processing method and apparatus | |
| CN101669117A (en) | Use matching images with shape descriptors | |
| WO2024120504A1 (en) | Data processing method and related device | |
| CN114359582B (en) | Small sample feature extraction method based on neural network and related equipment | |
| CN113626612A (en) | Prediction method and system based on knowledge graph reasoning | |
| CN106909560A (en) | POI sorting method | |
| CN115132324B (en) | Mental health prediction method and device, electronic device, and storage medium | |
| CN113298265B (en) | Heterogeneous sensor potential correlation learning method based on deep learning | |
| CN116883181B (en) | Financial service pushing method based on user portrait, storage medium and server | |
| CN111800535B (en) | Evaluation method, device, storage medium and electronic equipment for terminal operating state | |
| US20230419195A1 (en) | System and Method for Hierarchical Factor-based Forecasting | |
| CN115424060A (en) | Model training method, image classification method and device | |
| CN118133031A (en) | Abnormal event detection method, abnormal event detection model construction method and device | |
| CN117131963A (en) | Task processing method, neural network, training method, training device and training medium thereof | |
| CN112395314A (en) | Method, electronic device and computer readable medium for searching information | |
| CN118828145B (en) | Video abstract generation method and related device | |
| CN111800537B (en) | End-use status assessment method, device, storage medium and electronic equipment | |
| CN107766422A (en) | A kind of mapping method and equipment of data of registering | |
| Rahin et al. | Visualizing and Analyzing Human Activity Patterns through Graph-Based Methods | |
| CN116777065A (en) | Data prediction method, device, equipment and storage medium based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |