CN109946538B - Mobile equipment monitoring method and system based on magnetic induction signals - Google Patents
Mobile equipment monitoring method and system based on magnetic induction signals Download PDFInfo
- Publication number
- CN109946538B CN109946538B CN201910193710.3A CN201910193710A CN109946538B CN 109946538 B CN109946538 B CN 109946538B CN 201910193710 A CN201910193710 A CN 201910193710A CN 109946538 B CN109946538 B CN 109946538B
- Authority
- CN
- China
- Prior art keywords
- magnetic induction
- equipment
- extracted features
- induction data
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Telephone Function (AREA)
- Measuring Magnetic Variables (AREA)
Abstract
The invention relates to a mobile equipment monitoring method and a system based on magnetic induction signals, wherein the method comprises the following steps: collecting environmental magnetic induction data; preprocessing the magnetic induction data; performing characteristic extraction on the preprocessed magnetic induction data; inputting the extracted features into a pre-trained recognition model, and outputting a recognition result, wherein the recognition result comprises one or more of a user using the device, a device category and a currently running application. Compared with the prior art, the invention collects the magnetic induction signal generated by the intelligent mobile equipment when the intelligent mobile equipment works to complete the identification of the equipment type, the identification of the working state and the identification of the user through the magnetic field intensity sensor under the condition of not changing the hardware structure of the existing electrical equipment, thereby realizing the user authentication and the equipment monitoring and management of the mobile intelligent equipment at low cost.
Description
Technical Field
The invention relates to the field of bypass monitoring and management of intelligent mobile equipment, in particular to a mobile equipment monitoring method and system based on magnetic induction signals.
Background
Mobile devices play an irreplaceable role in our daily lives. It is predicted that by the end of 2020, there will be over 90 billion mobile phones, tablet computers and notebook computers. Capturing user device behavior, including knowing the type of device currently used by the user, which applications are running, and which user is using the device, is helpful in the fields of mobile device monitoring management and user authentication. However, a person may have multiple mobile devices, such as a smartphone and a laptop. Alternatively, a mobile device may be used by multiple users, for example, family members sharing a laptop computer. Conventional authentication methods cannot distinguish between different users. Therefore, a method of cross-platform user authentication is needed to understand user device behavior. A simple solution is to install a process monitor in each device and perform statistics in the cloud. Although this solution solves the cross-platform problem, user identification remains a difficult problem. Note that identification by a front facing camera is impractical for this problem, as turning the camera on all the time will cause the battery to drain in a short time. Fingerprinting is a promising method of identifying the behavior of user equipment. There are many other studies of off-channel to achieve this goal, such as the use of accelerometers, power consumption, and acoustic signal information. Accelerometer and power consumption information is too coarse to accurately identify the user, and the sound signal may be greatly disturbed by the environment.
The core of user authentication using magnetic induction signals needs to solve the time series classification problem. The traditional time series classification method artificially extracts time characteristics, and then inputs the time characteristics into a single classifier or an integrated classifier to generate output. The manual feature extraction is seriously dependent on the expert experience, and the accuracy of feature selection is difficult to ensure when the complicated scene of user authentication is involved. In addition, most of the traditional methods rely on features in the time domain, however, the features are easy to change along with time, and therefore the pre-trained model does not have good generalization performance.
Electromagnetic signals are a more suitable solution for user authentication and device management. We have observed that electromagnetic signals are a reflection of the computational intensity of mobile devices, for example, heavily loaded applications increase CPU power consumption and other power consumption, thereby providing the possibility for identification of applications. In addition, different users may have different user behavior habits when using the same device or even the same application, for example: the speed of typing, the usual operation, etc., resulting in corresponding changes in the electromagnetic signal. These also provide the possibility of using electromagnetic signals for user level identification. In recent years, deep neural networks have been used for time series classification tasks. Deep learning models automatically generate features and achieve the most advanced performance. Long term and short term memory networks have proven to be very good solutions to the time series problem.
Disclosure of Invention
The present invention is directed to a method and system for monitoring a mobile device based on magnetic induction signals, which overcome the above-mentioned drawbacks of the prior art.
The purpose of the invention can be realized by the following technical scheme:
a mobile device monitoring method based on magnetic induction signals comprises the following steps:
collecting environmental magnetic induction data;
preprocessing the magnetic induction data;
performing characteristic extraction on the preprocessed magnetic induction data;
inputting the extracted features into a pre-trained recognition model, and outputting a recognition result, wherein the recognition result comprises one or more of a user using the device, a device category and a currently running application.
The pre-processing the magnetic induction data comprises:
adopting a Gaussian filtering algorithm to carry out smooth filtering;
obtaining a frequency domain signal by adopting fast Fourier transform;
the principal components are extracted by principal component analysis.
The magnetic induction data after the pretreatment is subjected to feature extraction, and the feature extraction comprises the following steps:
carrying out time window division on the preprocessed magnetic induction data;
feature extraction is performed based on the time domain signal and the frequency domain signal of a single time window using a fully connected convolutional neural network model.
The identification result comprises a user using the equipment, equipment category and a currently running application;
the inputting of the extracted features into a pre-trained recognition model and the outputting of a recognition result specifically comprise:
inputting the extracted features into a pre-trained recognition model;
identifying and obtaining the equipment type according to the extracted features;
identifying and obtaining the currently running application according to the extracted features and the obtained equipment category;
and identifying and obtaining the user using the equipment according to the extracted features, the obtained equipment category and the currently running application.
The recognition model is a long-short-term memory full convolution neural network containing time domain and frequency domain features.
The utility model provides a mobile device monitored control system based on magnetic induction signal, includes SMD magnetic force sensor to and the host computer of being connected with SMD magnetic force sensor, the host computer includes:
the data preprocessing module is used for receiving environmental magnetic induction data sent by the surface mount type magnetic sensor and preprocessing the magnetic induction data;
the characteristic extraction module is used for extracting the characteristics of the preprocessed magnetic induction data;
and the recognition classification module is used for inputting the extracted features into a pre-trained recognition model and outputting a recognition result, wherein the recognition result comprises one or more of a user using the equipment, an equipment category and a currently running application.
The patch type magnetic sensor and the upper computer transmit data in a wireless communication mode.
The pre-processing the magnetic induction data comprises:
adopting a Gaussian filtering algorithm to carry out smooth filtering;
obtaining a frequency domain signal by adopting fast Fourier transform;
the principal components are extracted by principal component analysis.
The identification result comprises a user using the equipment, equipment category and a currently running application;
the inputting of the extracted features into a pre-trained recognition model and the outputting of a recognition result specifically comprise:
inputting the extracted features into a pre-trained recognition model;
identifying and obtaining the equipment type according to the extracted features;
identifying and obtaining the currently running application according to the extracted features and the obtained equipment category;
and identifying and obtaining the user using the equipment according to the extracted features, the obtained equipment category and the currently running application.
The magnetic induction data after the pretreatment is subjected to feature extraction, and the feature extraction comprises the following steps:
carrying out time window division on the preprocessed magnetic induction data;
feature extraction is performed based on the time domain signal and the frequency domain signal of a single time window using a fully connected convolutional neural network model.
Compared with the prior art, the invention has the following beneficial effects:
1) under the condition of not changing the hardware structure of the existing electrical equipment, the magnetic field intensity sensor is used for collecting magnetic induction signals generated by the intelligent mobile equipment when the intelligent mobile equipment works to complete the identification of the equipment type, the identification of the working state and the identification of a user, so that the user authentication and the equipment monitoring and management of the mobile intelligent equipment are realized at low cost.
2) In the preprocessing step, a frequency domain signal subjected to fast Fourier transform is adopted as a supplement, so that the defect that magnetic induction signal information on a time domain is easily interfered by the outside can be overcome, and the reliability is improved.
3) The principal component analysis is used for extracting the principal components, so that the interference of overhigh feature dimension on the system performance and the model generalization capability can be avoided.
4) The applicability is strong, can be applied to various types of intelligent mobile equipment.
5) The proposed frequency-domain-time-domain hierarchical deep learning model can be used to continuously and reliably distinguish applications and users, and this method performs well even if there are unknown applications and users.
Drawings
FIG. 1 is an overall architecture diagram of the present invention;
FIG. 2 is an inventive data preprocessing flow diagram;
FIG. 3 is a comparison graph of classification accuracy for various algorithmic models;
FIG. 4 is a graph comparing training times of various algorithm models.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
The method breaks through the limitations of traditional user authentication and equipment management, and the behavior of the user equipment is identified by tracking the magnetic signal sent by the mobile equipment. The invention also provides a TF-LSTM-FCN hierarchical classification model for continuously and reliably distinguishing applications and users. This method performs well even if there are unknown applications and users. The invention can accurately classify the current device type, running application program and user using the device, which is helpful in the fields of mobile device management and user authentication.
Specifically, the structure of this application needs to include SMD magnetic force sensor to and the host computer of being connected with SMD magnetic force sensor, the host computer includes:
the data preprocessing module is used for receiving environmental magnetic induction data sent by the surface mount type magnetic sensor and preprocessing the magnetic induction data;
the characteristic extraction module is used for extracting the characteristics of the preprocessed magnetic induction data;
and the recognition classification module is used for inputting the extracted features into a pre-trained recognition model and outputting a recognition result, wherein the recognition result comprises one or more of a user using the equipment, an equipment category and a currently running application.
The patch type magnetic sensor and the upper computer transmit data in a wireless communication mode, preferably in a WiFi mode, and the magnetic sensor can be configured with a battery to supply energy for the magnetic sensor due to low power consumption.
Wherein, carry out the preliminary treatment to magnetic induction data, include: adopting a Gaussian filtering algorithm to carry out smooth filtering; obtaining a frequency domain signal by adopting fast Fourier transform; the principal components are extracted by principal component analysis.
Specifically, because magnetic induction data collected by the patch type magnetic sensor has a lot of noises, the noises are mainly divided into manual operation, linear noise energy, electrode noise, external noise and internal noise. The noise can reduce the interference of noise signals to useful sensor signals by adjusting the position of the patch circuit board, the surrounding detection environment and some algorithms for filtering the noise. Therefore, data preprocessing is required before feature extraction, data in a window are normalized, and smooth filtering is performed by adopting a Gaussian filtering algorithm. In addition, experiments show that magnetic induction signal information in a time domain is easily interfered by the outside, so that the frequency domain signal subjected to fast Fourier transform is adopted as supplement; in order to avoid the interference of too high feature dimension on the system performance and the model generalization capability, the frequency domain information simultaneously extracts the main components by using principal component analysis.
Conventional methods of manually extracting features are often time consuming and require additional domain knowledge. Therefore, we turn to deep learning algorithms to automatically extract features. Existing research has shown that full convolutional networks perform well in terms of quality and efficiency of extracting features from time series data. Therefore, in both the time domain and the frequency domain, a full convolutional neural network is applied to extract features. As shown in fig. 2, the time domain and frequency domain data are added to the model together for feature extraction and classification. After we use the model to directly classify time series data, the three levels (device, application, and user levels) are not independent of each other due to the classification. For example, when classifying the type of device, the result is not affected by the application running on the device and the user of this device; the result of the classification application is strongly linked to the type of device. Therefore, we use hierarchical classification to improve the accuracy of each classification model.
The recognition model needs to be trained in advance, and as shown in fig. 1, the model needs to be trained and the prediction needs to be performed in real time for the recognition model. In the training model, the system first collects historical electromagnetic signals and labels information such as mobile device models, users and application types, and then feeds the labeled signals into the proposed hierarchical deep learning algorithm for supervised training. In the real-time prediction step, real-time magnetic induction signals are input into the trained model, and the model returns a prediction result, for example, a user A uses software C on equipment B, so that user authentication and equipment management of the mobile intelligent equipment are realized;
the classifier algorithm adopted by the method is a Frequency domain-Time domain Long Short Term Memory full convolutional neural network (TF-LSTM-FCN). The LSTM is a time-recursive neural network commonly used for processing time series problems, and due to a unique design structure, the LSTM is suitable for processing and predicting important events with very long intervals and delays in time series, and is generally used in the fields of pattern recognition, natural language processing and the like. The TF-LSTM-FCN model enhanced by using time domain and frequency domain information and extracted by using the automatic features of the full-convolution neural network can fully utilize information provided by original electromagnetic induction data, and complete intellectualization of user authentication and equipment management of the mobile intelligent equipment is realized.
The method and the device apply the trained real-time sensor data classifier model to practical application and verify the classification effect of the model. In a laboratory environment, data for 10 users using 30 different applications on 10 smart mobile devices was selected. The device types are respectively: apple, Hewlett packard, Association, Samsung, Del, Hongji, Huashuo and other notebook computers, and the application types comprise Microsoft Word, Excel, PPT, WeChat, QQ, Minecraft and the like. As shown in fig. 3 and 4, experiments have shown that the present system is able to correctly categorize 94.31% of users, 91.64% of applications, and 98.6% of mobile devices.
Claims (6)
1. A mobile device monitoring method based on magnetic induction signals is characterized by comprising the following steps:
the ambient magnetic induction data is collected and stored,
the magnetic induction data is pre-processed,
performing characteristic extraction on the preprocessed magnetic induction data,
inputting the extracted features into a pre-trained recognition model, and outputting a recognition result, wherein the recognition result comprises one or more of a user using the equipment, an equipment category and a currently running application;
the pre-processing the magnetic induction data comprises:
adopting a Gaussian filtering algorithm to carry out smooth filtering,
a frequency domain signal is obtained by using fast fourier transform,
extracting main components by adopting main component analysis;
the recognition result includes a user using the device, a device class and a currently running application,
the inputting of the extracted features into a pre-trained recognition model and the outputting of a recognition result specifically comprise:
inputting the extracted features into a pre-trained recognition model,
identifying and obtaining the equipment category according to the extracted features,
identifying and obtaining the current running application according to the extracted features and the obtained equipment category,
and identifying and obtaining the user using the equipment according to the extracted features, the obtained equipment category and the currently running application.
2. The method according to claim 1, wherein the performing feature extraction on the preprocessed magnetic induction data comprises:
carrying out time window division on the preprocessed magnetic induction data;
feature extraction is performed based on the time domain signal and the frequency domain signal of a single time window using a fully connected convolutional neural network model.
3. The method of claim 1, wherein the recognition model is a long-term short-term memory full convolution neural network including time domain and frequency domain features.
4. The utility model provides a mobile device monitored control system based on magnetic induction signal which characterized in that, includes SMD magnetic force sensor to and the host computer of being connected with SMD magnetic force sensor, the host computer includes:
the data preprocessing module is used for receiving the environmental magnetic induction data sent by the surface mount type magnetic sensor and preprocessing the magnetic induction data,
a characteristic extraction module for extracting the characteristics of the preprocessed magnetic induction data,
the recognition and classification module is used for inputting the extracted features into a pre-trained recognition model and outputting a recognition result, wherein the recognition result comprises one or more of a user using the equipment, an equipment category and a currently running application;
the pre-processing the magnetic induction data comprises:
adopting a Gaussian filtering algorithm to carry out smooth filtering,
a frequency domain signal is obtained by using fast fourier transform,
extracting main components by adopting main component analysis;
the recognition result includes a user using the device, a device class and a currently running application,
the inputting of the extracted features into a pre-trained recognition model and the outputting of a recognition result specifically comprise:
inputting the extracted features into a pre-trained recognition model,
identifying and obtaining the equipment category according to the extracted features,
identifying and obtaining the current running application according to the extracted features and the obtained equipment category,
and identifying and obtaining the user using the equipment according to the extracted features, the obtained equipment category and the currently running application.
5. The system according to claim 4, wherein the surface-mounted magnetic sensor and the upper computer transmit data in a wireless communication manner.
6. The system according to claim 4, wherein the feature extraction of the preprocessed magnetic induction data comprises:
carrying out time window division on the preprocessed magnetic induction data;
feature extraction is performed based on the time domain signal and the frequency domain signal of a single time window using a fully connected convolutional neural network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910193710.3A CN109946538B (en) | 2019-03-14 | 2019-03-14 | Mobile equipment monitoring method and system based on magnetic induction signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910193710.3A CN109946538B (en) | 2019-03-14 | 2019-03-14 | Mobile equipment monitoring method and system based on magnetic induction signals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109946538A CN109946538A (en) | 2019-06-28 |
| CN109946538B true CN109946538B (en) | 2020-06-19 |
Family
ID=67008846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910193710.3A Active CN109946538B (en) | 2019-03-14 | 2019-03-14 | Mobile equipment monitoring method and system based on magnetic induction signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109946538B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116611585A (en) * | 2023-07-18 | 2023-08-18 | 北京辰尧科技有限公司 | Mobile device management method and system and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105320966A (en) * | 2015-10-30 | 2016-02-10 | 东软集团股份有限公司 | Vehicle driving state recognition method and apparatus |
| CN109039643A (en) * | 2018-06-28 | 2018-12-18 | 中南民族大学 | A kind of sustainable method for authenticating user identity and system based on electromagnetic radiation |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096798B (en) * | 2011-01-25 | 2013-08-21 | 上海交通大学 | Signal processing method for electromagnetic induction state recognition |
| CN103425999A (en) * | 2013-08-27 | 2013-12-04 | 西安电子科技大学 | Brain cognitive state judgment method based on non-negative tensor projection operator decomposition algorithm |
| US11353965B2 (en) * | 2016-04-21 | 2022-06-07 | Carnegie Mellon University | System for enabling rich contextual applications for interface-poor smart devices |
| CN107273795B (en) * | 2017-05-04 | 2019-12-17 | 浙江大学 | Cross-device electromagnetic fingerprint database construction method and device based on machine learning |
| CN107292359B (en) * | 2017-05-04 | 2019-12-17 | 浙江大学 | Cross-equipment article marking method and device based on electromagnetic signal feature recognition |
| CN107220506A (en) * | 2017-06-05 | 2017-09-29 | 东华大学 | Breast cancer risk assessment analysis system based on deep convolutional neural network |
| CN107423756A (en) * | 2017-07-05 | 2017-12-01 | 武汉科恩斯医疗科技有限公司 | Nuclear magnetic resonance image sequence sorting technique based on depth convolutional neural networks combination shot and long term memory models |
| US9964607B1 (en) * | 2017-08-28 | 2018-05-08 | Seagate Technology Llc | Recognizing and identifying defect patterns on magnetic media |
| CN108196998A (en) * | 2018-01-02 | 2018-06-22 | 联想(北京)有限公司 | A kind of state identification method, mobile equipment and server |
| CN108596150A (en) * | 2018-05-10 | 2018-09-28 | 南京大学 | A kind of Activity recognition system and its working method excluding abnormal operation |
| CN109344809A (en) * | 2018-11-21 | 2019-02-15 | 上海交通大学 | Domestic electric appliance intelligent management system based on magnetic strength induction signal |
-
2019
- 2019-03-14 CN CN201910193710.3A patent/CN109946538B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105320966A (en) * | 2015-10-30 | 2016-02-10 | 东软集团股份有限公司 | Vehicle driving state recognition method and apparatus |
| CN109039643A (en) * | 2018-06-28 | 2018-12-18 | 中南民族大学 | A kind of sustainable method for authenticating user identity and system based on electromagnetic radiation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109946538A (en) | 2019-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110364144B (en) | Speech recognition model training method and device | |
| CN103811003B (en) | A kind of audio recognition method and electronic equipment | |
| CN107147618A (en) | A kind of user registering method, device and electronic equipment | |
| KR101939683B1 (en) | Apparatus and method for recognizing user activity | |
| CN104867495A (en) | Sound Symbol Detection Of Context Sensing | |
| US20190043489A1 (en) | Acoustic event detector with reduced resource consumption | |
| CN109346061A (en) | Audio-frequency detection, device and storage medium | |
| CN111599371A (en) | Voice adding method, system, device and storage medium | |
| CN110648669B (en) | Multi-frequency shunt voiceprint recognition method, device and system and computer readable storage medium | |
| CN115798459B (en) | Audio processing method and device, storage medium and electronic equipment | |
| CN114760172A (en) | Method and device for identifying radio frequency baseband comprehensive characteristic signal | |
| CN111323783A (en) | Scene recognition method and device, storage medium and electronic equipment | |
| CN109946538B (en) | Mobile equipment monitoring method and system based on magnetic induction signals | |
| TW201801066A (en) | Audio identification method and device | |
| CN114898527A (en) | Wearable old man falling detection system and method based on voice assistance | |
| CN114121025A (en) | Voiceprint fault intelligent detection method and device for substation equipment | |
| CN112733927A (en) | Fan blade sound boundary positioning method based on one-dimensional convolutional neural network and storage device | |
| CN112863548A (en) | Method for training audio detection model, audio detection method and device thereof | |
| CN111933180A (en) | Audio splicing detection method and system, mobile terminal and storage medium | |
| CN118411984A (en) | Voice wake-up method and device of MEMS speaker, electronic equipment and storage medium | |
| Ji et al. | TFAM-AAE-U k: A Dual-Metric Spectrum Anomaly Detection Algorithm | |
| Liu et al. | End-to-end elevator voice command recognition system based on improved convolutional neural network | |
| CN114141246A (en) | Method for recognizing speech, method and device for training model | |
| CN115223551A (en) | Voice awakening method and system based on voice similarity matching | |
| CN115995014A (en) | Method for detecting loudspeaker monomer, method for detecting audio frequency and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 200030 Dongchuan Road, Minhang District, Minhang District, Shanghai Applicant after: Shanghai Jiaotong University Address before: 200030 Huashan Road, Shanghai, No. 1954, No. Applicant before: Shanghai Jiaotong University |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |