CN113298265A - Heterogeneous sensor potential correlation learning method based on deep learning - Google Patents
Heterogeneous sensor potential correlation learning method based on deep learning Download PDFInfo
- Publication number
- CN113298265A CN113298265A CN202110561563.8A CN202110561563A CN113298265A CN 113298265 A CN113298265 A CN 113298265A CN 202110561563 A CN202110561563 A CN 202110561563A CN 113298265 A CN113298265 A CN 113298265A
- Authority
- CN
- China
- Prior art keywords
- data
- sensor
- correlation
- learning
- sensors
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000013135 deep learning Methods 0.000 title claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000008447 perception Effects 0.000 claims description 12
- 238000013136 deep learning model Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000001364 causal effect Effects 0.000 claims description 3
- 230000005484 gravity Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 abstract description 2
- 238000010219 correlation analysis Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004587 chromatography analysis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
- Telephone Function (AREA)
Abstract
The invention relates to a method for analyzing potential relevance of multiple heterogeneous sensors based on deep learning. The method firstly needs to collect the required sensor data from the intelligent device, and the data of the sensors needs to be collected by using the corresponding sampling rate. Storing the collected sensor data into a specified file; secondly, in order to analyze potential correlation among various sensors, data of one sensor is used as a reference, and then the method is used for analyzing the proportion of influence between data of other sensors corresponding to the data of the sensor; finally, the correlation relationship among the various sensors is obtained by analyzing the various sensors as different reference objects, so that the potential correlation representation among the various heterogeneous sensors is obtained.
Description
Technical Field
The invention belongs to the technical field of data processing, relates to a deep learning-based heterogeneous sensor potential correlation learning method, and particularly relates to the field of data acquisition and analysis and the field of data correlation of a sensor of a smart phone used by a user.
Background
In recent years, thanks to the development of technologies such as intelligent sensing and pervasive computing, a cooperative sensing technology widely applied to network convergence becomes a research hotspot. With the rapid development of embedded devices, wireless sensor networks, internet of things and intelligent mobile terminals, a universal intelligent sensing system with integrated sensing, computing and communication capabilities is gradually integrated into the daily life of the society, and the capability of intelligently sensing and acquiring data is remarkably enhanced.
The perception capability obtained by using a single perception device is limited, and in recent years, popularization of various intelligent devices, particularly smart phones, enables common people to share various information such as geographic positions, surrounding environments and the like in real time. The performance of the processor is continuously enhanced, and the smart phone is greatly innovated. Increasingly complex sensors such as sensors capable of measuring acceleration, gyroscope sensors, gravity sensors and direction sensors become standard configurations of smart phones. The sensors enable each individual carrying a mobile phone to become a terminal with strong sensing capability, so that a great number of individuals carrying intelligent terminals can form a coordinator sensing network to observe, track and the like specific targets. Learning of the heterogeneous sensor potential relevance is required.
The document "Rajesh G, Charturedi A. correlation analysis and statistical characterization of heterologous sensor data in environmental sensor Networks [ J ]. Computer Networks, 2019" proposes four classical and robust correlation coefficient measurement methods. The paper utilizes the correlation between sensor modalities to complete the subsequent sensor data interpolation and new modality data prediction, and defines and expresses the correlation between modalities by measuring the correlation coefficient between different multi-modality variables.
The document "Jiang M, Gao X, An H, et al.Reconstructing complex network for chromatography of the time-varying evaluation analysis viewer of multi-varying time series [ J ]. Scientific Reports,2017,7 (1): 10486-. The method has the potential of analyzing the multivariate time sequence and provides important information for investors and decision makers;
the document "Gupta S, Buriro a, Crispo b. driverauth: a Risk-based Multi-modal Biometric-based Driver Authentication Scheme [ J ] Computers & Security, 2019' through learning and utilizing the correlation among a plurality of heterogeneous sensors carried in smart phone equipment, a novel Multi-modal Biometric Authentication solution based on risks is realized, and the aim of ensuring the certification service of a network car booking Driver to be safer and more reliable is achieved.
The above documents briefly introduce the existing correlation analysis method and its main application scenario, and the existing correlation analysis method mainly represents the correlation by calculating the corresponding correlation coefficient. But for heterogeneous sensor devices, the data it produces is heterogeneous time series data. For the correlation analysis between the heterogeneous time series data in a large scale, the existing correlation analysis method based on correlation coefficient calculation cannot meet the requirement.
Disclosure of Invention
Technical problem to be solved
In order to solve the problem that the traditional correlation analysis method based on correlation coefficient calculation cannot effectively perform correlation analysis on the perception data of the heterogeneous sensor, the invention provides a heterogeneous sensor potential correlation learning method based on deep learning. The method is used for carrying out potential correlation analysis on a large amount of heterogeneous time sequence data generated by various sensors, and a real-time correlation analysis task is realized by utilizing a deep learning technology. The invention also optimizes the traditional time convolution network model and creatively realizes the task of analyzing the correlation of the time sequence data.
Technical scheme
A heterogeneous sensor potential correlation learning method based on deep learning is characterized by comprising the following steps:
step 1: acquiring perception data of a terminal user by using a sensor;
step 2: storing and preprocessing the acquired sensing data of the sensor;
and step 3: executing a data alignment task on the sensor data file of the acquired data;
and 4, step 4: taking the sensor data after executing the data alignment task as input data of a deep learning model, wherein the output data of the deep learning model is data of any sensor; after learning, the deep learning model returns a two-dimensional matrix to represent the correlation between various sensors which are input currently and output sensors;
and 5: learning all the sensor data as output data of the model, and finally summarizing the corresponding relations among all the sensors to generate a correlation two-dimensional matrix among all the sensors;
step 6: a two-dimensional matrix of all sensor correlations is visualized.
Preferably: the terminal in the step 1 is a smart phone.
Preferably: the sensor in the step 1 is a gyroscope sensor, a gravity sensor or a direction sensor.
Preferably: the file type of the stored sensor data in the step 2 is a CSV file.
Preferably: the data alignment task in step 3 is specifically: selecting the sensor S with the longest data lengthkThen, S iskAnd performing an alignment task on the data of the other sensors as a reference element of the data, selecting the data which is collected by the sensor for the last time to be supplemented when the alignment task is executed, and finally, all the sensors have data with relevant lengths after the execution of the data-optimal task is finished.
Preferably: the deep learning model in the step 4 comprises causal convolution, dilation convolution, residual connection and attention mechanism.
Preferably: the correlation in step 4 is specifically as follows: the decimal between 0 and 1 is used for representation, 0 represents the minimum correlation, and 1 represents the highest correlation.
Preferably: step 6 is visualized using thermodynamic diagrams.
Advantageous effects
The heterogeneous sensor correlation learning method based on deep learning can be used for completing the task of analyzing the correlation between heterogeneous sensors. For a large amount of time sequence data generated by heterogeneous sensors, the method can learn the correlation among the data through an improved time convolution network model, and the improved time convolution network model can express the correlation in a digital form through an attention mechanism of the improved time convolution network model, and finally, potential correlation relations among the heterogeneous sensors can be visually observed through a data visualization technology. The perception capability of the heterogeneous sensors can be fully exerted in intelligent perception and pervasive computing tasks by using the intuitive correlation results, the perception tasks are completed in a more efficient collaborative perception mode, cost reduction and efficiency improvement of the perception tasks are realized, and a plurality of complex user behavior recognition, user context perception and other high-level perception tasks are completed in a more energy-saving mode.
Drawings
FIG. 1 is a flowchart of a deep learning-based method for learning potential correlations of heterogeneous sensors according to an embodiment of the present invention
FIG. 2 is a flow chart of the execution of data alignment tasks during data pre-processing phase in an embodiment of the present invention
FIG. 3 is a diagram of a deep learning-based relevance latent learning neural network model structure constructed in an embodiment of the present invention
Detailed Description
The invention will now be further described with reference to the following examples and drawings:
the method comprises the steps that firstly, sensing data of all sensors are acquired from intelligent equipment carrying a plurality of heterogeneous sensors, and specifically, the same sampling frequency is used for acquiring all acquired data of the heterogeneous sensors; the neural network model is then used to learn the correlation between these sensors, and in particular the inputs and outputs of the model, here the corresponding sensor data, need to be specified. Meanwhile, in order to train the neural network model better, corresponding data is divided into two parts, namely training and testing. Secondly, in order to learn the correlation between the heterogeneous sensors, the influence relationship of the input data relative to the output data needs to be extracted in the model. Therefore, the part is specifically set in the model to learn the correlations, i.e., a matrix is used to hold the correlations. In order to represent the correlation between heterogeneous sensors, a decimal between 0 and 1 is used for representation, 0 represents the minimum correlation, and 1 represents the highest correlation. Next, in order to learn the potential correlation between a plurality of heterogeneous sensors, it is necessary to use all the heterogeneous sensors as the output of the model, and to sequentially input the data of the other sensors, and then to perform model learning and recording a plurality of times. Finally, the learned correlation coefficient data among the various heterogeneous sensor data are sorted, and finally the data are expressed as a two-dimensional matrix. The two-dimensional matrix records the correlations between all heterogeneous sensors and other heterogeneous sensors. Finally, in order to better show the potential correlation among the sensors, the invention also adopts a visualization strategy to express the potential correlation among the heterogeneous sensors learned by the neural network.
In order to realize the task, the invention adopts the following technical scheme: the heterogeneous sensor potential correlation learning method based on deep learning comprises the following steps:
step one, a user uses a portable smart phone to carry out daily activities, and meanwhile, a sensor mounted on the smart phone starts to collect data sensed currently.
And step two, sending the acquired sensing data of the sensor to a server side for storage, wherein the acquired sensor data file type is a CSV file. And after all the files are uploaded, the data acquisition task of the smart phone end is finished.
And thirdly, carrying out data preprocessing operation on the acquired sensor data file to reduce the influence of invalid data. And deleting the sensor files without the acquired data by data preprocessing, and only leaving the sensor data files with the acquired data.
Step four, executing data alignment task on the sensor data file of the acquired data, and executing data alignment taskThe execution flow chart of (2) is as shown in fig. 2. Selecting the sensor S with the longest data lengthk. Then the S iskAs a reference element of the data, the data of the remaining sensors are subjected to an alignment task. And when the alignment task is executed, the data acquired by the sensor at the last time is selected for completion, and finally all the sensors have data with relevant length after the execution of the data completion task is finished.
And step five, taking the sensor data after the data alignment task is executed as input data of the deep learning model, wherein the output data of the deep learning model is data of any sensor. The structure diagram of the deep learning based neural network model is shown in fig. 3, and the model comprises four components: causal convolution, dilation convolution, residual concatenation and attention mechanism. Consists of two layers: a time convolutional network layer, an attention layer. After the input and output data of the model are specified, a model learning task is started, and after the model is learned, a correlation matrix is returned to represent the potential correlation relationship existing between the currently input sensor and the currently specified output sensor.
And step six, selecting other sensors as output data of the model to perform a task of learning the potential correlation between the sensors and the input sensors, and finally performing potential correlation learning by using all the sensors as the output data of the model. And summarizing the learned correlation matrix, respectively displaying the potential correlation between each sensor relative to other sensors, and generating a correlation two-dimensional matrix among all the sensors.
Finally, a two-dimensional matrix visualization thermodynamic diagram representing all sensor correlations is output as a final result.
Claims (8)
1. A heterogeneous sensor potential correlation learning method based on deep learning is characterized by comprising the following steps:
step 1: acquiring perception data of a terminal user by using a sensor;
step 2: storing and preprocessing the acquired sensing data of the sensor;
and step 3: executing a data alignment task on the sensor data file of the acquired data;
and 4, step 4: taking the sensor data after executing the data alignment task as input data of a deep learning model, wherein the output data of the deep learning model is data of any sensor; after learning, the deep learning model returns a two-dimensional matrix to represent the correlation between various sensors which are input currently and output sensors;
and 5: learning all the sensor data as output data of the model, and finally summarizing the corresponding relations among all the sensors to generate a correlation two-dimensional matrix among all the sensors;
step 6: a two-dimensional matrix of all sensor correlations is visualized.
2. The deep learning-based heterogeneous sensor potential correlation learning method according to claim 1, wherein the terminal in step 1 is a smart phone.
3. The method for learning the potential correlation of the heterogeneous sensor based on the deep learning of claim 1, wherein the sensor in the step 1 is a gyroscope sensor, a gravity sensor or an orientation sensor.
4. The method for learning the potential relevance of the heterogeneous sensor based on deep learning of claim 1, wherein the file type of the saved sensor data in step 2 is a CSV file.
5. The method for learning the potential relevance of the heterogeneous sensor based on the deep learning of claim 1, wherein the data alignment task in the step 3 is specifically: selecting the sensor S with the longest data lengthkThen, S iskPerforming alignment tasks on the data of the other sensors as reference elements of the data, selecting the data which is acquired by the sensor for the last time for completing the alignment tasks when the alignment tasks are executed, and finally performing the data completing tasksAll sensors have data of the relevant length after the execution is finished.
6. The method for learning the potential correlation of the heterogeneous sensor based on the deep learning of claim 1, wherein the deep learning model in the step 4 comprises a causal convolution, an expansion convolution, a residual join and an attention mechanism.
7. The method for learning the potential correlation of the heterogeneous sensor based on the deep learning of claim 1, wherein the correlation relationship in the step 4 is specifically as follows: the decimal between 0 and 1 is used for representation, 0 represents the minimum correlation, and 1 represents the highest correlation.
8. The deep learning based heterogeneous sensor potential correlation learning method of claim 1, wherein step 6 is visualized by using thermodynamic diagrams.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110561563.8A CN113298265B (en) | 2021-05-22 | 2021-05-22 | Heterogeneous sensor potential correlation learning method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110561563.8A CN113298265B (en) | 2021-05-22 | 2021-05-22 | Heterogeneous sensor potential correlation learning method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298265A true CN113298265A (en) | 2021-08-24 |
| CN113298265B CN113298265B (en) | 2024-01-09 |
Family
ID=77324026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110561563.8A Active CN113298265B (en) | 2021-05-22 | 2021-05-22 | Heterogeneous sensor potential correlation learning method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298265B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220407769A1 (en) * | 2021-06-16 | 2022-12-22 | Ironwood Cyber Inc. | Control system anomaly detection using neural network consensus |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103018181A (en) * | 2012-12-14 | 2013-04-03 | 江苏大学 | Soft measurement method based on correlation analysis and ELM neural network |
| CN105760507A (en) * | 2016-02-23 | 2016-07-13 | 复旦大学 | Cross-modal subject correlation modeling method based on deep learning |
| CN107038506A (en) * | 2017-05-09 | 2017-08-11 | 华东师范大学 | A kind of factory's intelligent early-warning method |
| US20190228272A1 (en) * | 2018-01-23 | 2019-07-25 | Intelligent Fusion Technology, Inc | Methods, systems and media for joint manifold learning based heterogenous sensor data fusion |
| CN110209822A (en) * | 2019-06-11 | 2019-09-06 | 中译语通科技股份有限公司 | Sphere of learning data dependence prediction technique based on deep learning, computer |
| US20190293462A1 (en) * | 2018-03-23 | 2019-09-26 | Industry-Academic Cooperation Foundation, Dankook University | Apparatus and method for processing multi-type sensor signal on the basis of multi-modal deep learning |
| CN112116019A (en) * | 2020-09-27 | 2020-12-22 | 四川大学 | Multi-sensor Vine Copula heterogeneous information decision fusion method |
-
2021
- 2021-05-22 CN CN202110561563.8A patent/CN113298265B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103018181A (en) * | 2012-12-14 | 2013-04-03 | 江苏大学 | Soft measurement method based on correlation analysis and ELM neural network |
| CN105760507A (en) * | 2016-02-23 | 2016-07-13 | 复旦大学 | Cross-modal subject correlation modeling method based on deep learning |
| CN107038506A (en) * | 2017-05-09 | 2017-08-11 | 华东师范大学 | A kind of factory's intelligent early-warning method |
| US20190228272A1 (en) * | 2018-01-23 | 2019-07-25 | Intelligent Fusion Technology, Inc | Methods, systems and media for joint manifold learning based heterogenous sensor data fusion |
| US20190293462A1 (en) * | 2018-03-23 | 2019-09-26 | Industry-Academic Cooperation Foundation, Dankook University | Apparatus and method for processing multi-type sensor signal on the basis of multi-modal deep learning |
| CN110209822A (en) * | 2019-06-11 | 2019-09-06 | 中译语通科技股份有限公司 | Sphere of learning data dependence prediction technique based on deep learning, computer |
| CN112116019A (en) * | 2020-09-27 | 2020-12-22 | 四川大学 | Multi-sensor Vine Copula heterogeneous information decision fusion method |
Non-Patent Citations (2)
| Title |
|---|
| G. RAJESH等: "Correlation analysis and statistical characterization of heterogeneous sensor data in environmental sensor networks", COMPUTER NETWORKS * |
| 丁小欧;于晟健;王沐贤;王宏志;高宏;杨东华;: "基于相关性分析的工业时序数据异常检测", 软件学报, no. 03 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220407769A1 (en) * | 2021-06-16 | 2022-12-22 | Ironwood Cyber Inc. | Control system anomaly detection using neural network consensus |
| US11546205B1 (en) * | 2021-06-16 | 2023-01-03 | Ironwood Cyber Inc. | Control system anomaly detection using neural network consensus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298265B (en) | 2024-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210513B (en) | Data classification method and device and terminal equipment | |
| CN111914113B (en) | Image retrieval method and related device | |
| WO2024041479A1 (en) | Data processing method and apparatus | |
| CN112509690A (en) | Method, apparatus, device and storage medium for controlling quality | |
| EP4273754A1 (en) | Neural network training method and related device | |
| CN113538070B (en) | User life value cycle detection method and device and computer equipment | |
| CN110909578A (en) | Low-resolution image recognition method and device and storage medium | |
| CN116775918B (en) | Cross-modal retrieval method, system, equipment and medium based on complementary entropy contrast learning | |
| CN113705534A (en) | Behavior prediction method, behavior prediction device, behavior prediction equipment and storage medium based on deep vision | |
| CN115132324A (en) | Mental health prediction method and device, electronic equipment and storage medium | |
| CN113298265B (en) | Heterogeneous sensor potential correlation learning method based on deep learning | |
| CN114334036A (en) | Model training method, related device, equipment and storage medium | |
| CN116186295B (en) | Attention-based knowledge graph link prediction method, attention-based knowledge graph link prediction device, attention-based knowledge graph link prediction equipment and attention-based knowledge graph link prediction medium | |
| CN117217812A (en) | User behavior prediction method and device, computer equipment and storage medium | |
| CN112948763A (en) | Method and device for predicting quantity of component, electronic equipment and storage medium | |
| CN115424060A (en) | Model training method, image classification method and device | |
| CN111800535A (en) | Terminal running state evaluation method and device, storage medium and electronic equipment | |
| CN116720517B (en) | Search word component recognition model construction method and search word component recognition method | |
| CN115701866B (en) | E-commerce platform risk identification model training method and device | |
| CN111800537B (en) | Terminal use state evaluation method and device, storage medium and electronic equipment | |
| CN117709271A (en) | Simulation circuit netlist division method and system based on reinforcement learning | |
| CN116777065A (en) | Data prediction method, device, equipment and storage medium based on artificial intelligence | |
| Uniyal et al. | Employing DL-Based Algorithm for Gender and Age Identification | |
| CN117252713A (en) | Risk identification method, device and equipment for new energy vehicle and storage medium | |
| CN118133031A (en) | Abnormal event detection method, abnormal event detection model construction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |