CN116382472A - Multi-mode data-based wearable remote real-time monitoring system and method for electric power operators - Google Patents
Multi-mode data-based wearable remote real-time monitoring system and method for electric power operators Download PDFInfo
- Publication number
- CN116382472A CN116382472A CN202310197634.XA CN202310197634A CN116382472A CN 116382472 A CN116382472 A CN 116382472A CN 202310197634 A CN202310197634 A CN 202310197634A CN 116382472 A CN116382472 A CN 116382472A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- module
- unit
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 98
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000006399 behavior Effects 0.000 claims description 96
- 238000000605 extraction Methods 0.000 claims description 42
- 238000010606 normalization Methods 0.000 claims description 36
- 230000004927 fusion Effects 0.000 claims description 34
- 238000007781 pre-processing Methods 0.000 claims description 32
- 230000005856 abnormality Effects 0.000 claims description 31
- 239000007789 gas Substances 0.000 claims description 31
- 230000001133 acceleration Effects 0.000 claims description 30
- 238000011176 pooling Methods 0.000 claims description 30
- 238000012545 processing Methods 0.000 claims description 30
- 230000007613 environmental effect Effects 0.000 claims description 27
- 230000010365 information processing Effects 0.000 claims description 27
- 239000013598 vector Substances 0.000 claims description 27
- 230000017105 transposition Effects 0.000 claims description 24
- 238000012937 correction Methods 0.000 claims description 20
- 210000002569 neuron Anatomy 0.000 claims description 18
- 230000005284 excitation Effects 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 16
- 238000001125 extrusion Methods 0.000 claims description 16
- 230000005684 electric field Effects 0.000 claims description 14
- 230000036760 body temperature Effects 0.000 claims description 12
- 230000005540 biological transmission Effects 0.000 claims description 11
- 230000006698 induction Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 10
- 239000008280 blood Substances 0.000 claims description 10
- 210000004369 blood Anatomy 0.000 claims description 10
- 229910052760 oxygen Inorganic materials 0.000 claims description 10
- 239000001301 oxygen Substances 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 8
- 230000005611 electricity Effects 0.000 claims description 8
- 230000015572 biosynthetic process Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000003786 synthesis reaction Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 238000012986 modification Methods 0.000 claims description 6
- 230000004048 modification Effects 0.000 claims description 6
- 230000002194 synthesizing effect Effects 0.000 claims description 6
- 231100000331 toxic Toxicity 0.000 claims description 5
- 230000002588 toxic effect Effects 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 5
- 206010000117 Abnormal behaviour Diseases 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000007405 data analysis Methods 0.000 claims description 4
- 238000012644 addition polymerization Methods 0.000 claims description 3
- 239000003814 drug Substances 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 17
- 230000000694 effects Effects 0.000 description 6
- RAHZWNYVWXNFOC-UHFFFAOYSA-N Sulphur dioxide Chemical compound O=S=O RAHZWNYVWXNFOC-UHFFFAOYSA-N 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 2
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical compound [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 description 2
- 229910002091 carbon monoxide Inorganic materials 0.000 description 2
- 210000004027 cell Anatomy 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910002092 carbon dioxide Inorganic materials 0.000 description 1
- 239000001569 carbon dioxide Substances 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000013186 photoplethysmography Methods 0.000 description 1
- 230000037081 physical activity Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B5/02055—Simultaneously evaluating both cardiovascular condition and temperature
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1112—Global tracking of patients, e.g. by using GPS
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/6802—Sensor mounted on worn items
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Abstract
The invention relates to a multimode data-based power operator wearable remote real-time monitoring system and method. The monitoring system acquires the behavior, environment and positioning information of the user through the wearable equipment, and carries out comprehensive judgment of multi-azimuth safety risks; meanwhile, a multi-channel attention model network based on transposed coding is adopted, so that multi-dimensional potential information in user behavior data can be obtained; the practicability, reliability and robustness of the system are enhanced, and the user behavior recognition classification and danger early warning can be better realized.
Description
Technical Field
The invention relates to a system and a method for monitoring power operators in a wearable remote real-time manner based on multimode data, and belongs to the technical field of artificial intelligence.
Background
Technologies for recording daily behaviors of a human body or monitoring the physical health of a user by using products and electronic devices having specific functions, which are worn on the user, have been proposed in the middle of the earliest last century. With the development of sensor technology, various wearable sensor layers with lighter weight and smaller volume are endless, such as an inertial measurement unit (Inertial Measurement Unit, IMU) sensor, acceleration data and angular acceleration data can be acquired simultaneously, and a photoelectric volume pulse wave (Photo Plethysmo Graphy, PPG) sensor can measure physiological parameters of heart rate and blood oxygen saturation of a human body.
By means of the internet of things and the cloud platform technology, the wearable device can conduct real-time signal transmission and information communication. Through establishing the connection between the wearable equipment and the cloud platform, data collected by the wearable equipment can be rapidly transmitted to the cloud, and the human monitoring model can be conveniently analyzed and alarmed at the cloud.
At present, human behavior monitoring work based on wearable equipment mainly comprises the following contents: various effective human body activity data are collected, effective characteristic values are extracted, human body behavior recognition and classification methods are researched, a human body behavior monitoring system based on wearable equipment is built, and the like. Most wearable device-based human behavior monitoring methods research proceeds around one or more problems therein.
The strategy method for processing the multi-mode physiological signals mainly comprises the following steps: and the complementary information is fused by using the mutual support of signals of different categories, so that the identification effect is finally effectively improved. According to the existing researches, the mode fusion modes can be roughly divided into 4 types, namely data level fusion (sensing layer fusion), feature level fusion, decision level fusion and model level fusion.
In a special power operation scene, more factors, such as portability of wearable equipment, diversity of collected data types, recognition scenes of a detection algorithm and the like, need to be considered. In order to improve the applicability of the wearable remote real-time monitoring system for electric power operators in a complex environment, the wearable equipment needs to collect multi-mode information such as positioning data, near-electric induction and physiological information as widely as possible. In addition, in order to improve portability of the collecting device, various functions are integrated as much as possible to reduce the size of the device while reducing power consumption to extend the use time. And the data is transmitted to the cloud platform through a network, and analysis and alarm are carried out on the data while the reliability of data transmission is ensured. A cloud platform with management capability is built, real-time monitoring is carried out on workers, useful information is provided, an electric company is helped to improve the operation flow, and the working efficiency is improved.
Disclosure of Invention
Aiming at the special monitoring requirements of an electric power scene and the problems of incomplete feature extraction and low utilization rate in the scene, the invention provides a behavior recognition method based on multi-mode feature extraction and coding, which acts on a wearable remote real-time monitoring system for electric power operators.
The multi-mode sensor data used by the invention is acquired by wearable sensors worn at different positions of a human body, and the multi-mode sensor data is acquired by a plurality of wearable sensors at the same time, wherein the multi-mode sensor data comprises an accelerometer sensor, a gyroscope sensor, an environmental electric field sensor, an environmental gas sensor and a positioning sensor, can acquire data of five modes of acceleration, angular velocity, an environmental electric field, environmental gas and positioning information at the same time, and can be expanded to a plurality of modes.
The invention adopts a convolution network model, and based on an extrusion excitation (Squeeze and Excitation, SE) module and a transposition coding module, a multichannel feature extraction and anomaly monitoring module is realized, physiological information is fully captured, and a higher recognition rate is obtained.
The invention builds a complete remote monitoring system, which comprises the following steps: wearable equipment, cloud platform, webpage end.
The wearable device has the functions of data collection, data transmission and instruction reception. By receiving commands from the cloud, data is collected using a variety of sensors and transmitted to the cloud platform.
The cloud platform receives data from the wearable equipment, processes the multi-mode data through data preprocessing and data analysis, completes monitoring and alarming, and sends a visual result to a webpage end.
The webpage end has the functions of data display and administrator operation. The visual result obtained by processing the data collected by the wearable equipment and the alarm information of the user are displayed, and an administrator corrects the data and issues a command to the wearable equipment through webpage end operation.
The invention sets various collected information such as near-electric induction, dangerous gas, location-Based Service (LBS) positioning and the like according to the working characteristics of electric personnel, and provides corresponding processing algorithms for various collected data.
Summary of the invention:
the invention provides a real-time behavior recognition method based on feature extraction of transposed codes and a wearable remote real-time monitoring system of electric power operators, which comprise an operator real-time information acquisition module, a multi-mode data preprocessing module, a feature extraction and early warning module based on the transposed codes and a data display and correction module which are sequentially connected. Firstly, after receiving a command, collecting various information by sensor equipment in a real-time information collecting module; the multi-mode data real-time transmission module realizes real-time data transmission in a wireless transmission mode and transmits data to a cloud database through a TCP/IP protocol; judging the data type through a cloud server program, and respectively processing environment information, behavior data and positioning data; the environment information exceeds the alarm value for early warning, the positioning data judges the type of the positioning data, if the type of the positioning data is LBS data, the data decoding is carried out, and then the positioning data is transmitted into a multi-source data fusion positioning module; for behavior data, firstly, preprocessing the behavior data, namely denoising, normalizing, data fusion and intercepting based on time sequences, and finally constructing and training a behavior recognition model, and then carrying out behavior real-time recognition; and finally, displaying all the early warning information obtained in the steps in real time and providing the information to a manager.
The invention provides a feasible scheme for the electric power staff monitoring system based on the wearable equipment, can monitor the behavior category of the user in real time, overcomes the defects of single sensor data and a simple deep learning network in the aspects of self-adaption and special behavior classification, further extracts comprehensive and effective characteristics, improves the characteristic utilization rate, can be used for monitoring and managing electric power system operators, provides a complete flow, and fills the blank of the monitoring system for special industries.
The technical scheme of the invention is as follows:
the utility model provides a wearable remote real-time monitoring system of electric power operation personnel based on multimode data, includes that the operation personnel real-time information acquisition module, multimode data preprocessing module, feature extraction and early warning module, data show and correction module based on transposition code that connect gradually;
the real-time information acquisition module of the operating personnel is used for: collecting real-time information through the wearable equipment, wherein the real-time information comprises environment information, physiological information and positioning information;
the multi-mode data preprocessing module is used for: receiving real-time information from the wearable device and preprocessing the real-time information;
The feature extraction and early warning module based on transpose coding is used for: processing environment, physiological and positioning information data, and performing human behavior recognition through a feature extraction algorithm based on transposed encoding;
the data display and correction module is used for: receiving data from a data identification and early warning module, synthesizing various data, and performing visual display and alarm on a webpage end; meanwhile, an administrator operation interface is provided at the webpage end for an administrator to send instructions and correction data.
According to the invention, the real-time information acquisition module of the operator comprises an environment information acquisition unit, a physiological information acquisition unit, a positioning information acquisition unit and a data transmission unit which are connected in parallel;
the environment information acquisition unit is used for: acquiring environmental information; the physiological information acquisition unit is used for: acquiring physiological information; the positioning information acquisition unit is used for: acquiring positioning information; the data transmission unit is used for: and remotely transmitting the collected behavior data of the user to a database of the cloud server in real time through a TCP/IP protocol.
Further preferably, the environmental information acquisition unit comprises an air quality detection unit and a near electricity induction unit;
The air quality detection unit is used for: reading environmental data in the air, including the concentration of a plurality of toxic and harmful gases;
the near-electricity induction unit is used for: reading the electric field intensity near the human body, and realizing power plant monitoring and alarming;
further preferably, the physiological information acquisition unit comprises an acceleration sensor, an angular velocity sensor, a temperature sensor and a PPG sensor;
the acceleration sensor is used for acquiring acceleration data of X, Y, Z triaxial; the angular velocity sensor is used for acquiring X, Y, Z triaxial angular velocity data of the angular velocity sensor; the temperature sensor is used for acquiring body temperature data; the PPG sensor is used for measuring heart rate and blood oxygen saturation;
further preferably, the positioning information acquisition unit comprises a global positioning system (Global Positioning System, GPS) and a Wi-Fi base station information receiving system;
the global positioning system is used for acquiring GPS signals and service signals LBS; the Wi-Fi base station information receiving system is configured to obtain a media access control address (Media Access Control Address, MAC) and a signal strength.
Further preferably, the pretreatment comprises: and denoising filtering, normalization, data fusion and time sequence based interception are sequentially carried out.
According to the invention, the multi-mode data preprocessing module comprises a data receiving unit, a data preprocessing unit and a data normalizing unit which are connected in sequence;
the data receiving unit is used for receiving various data from the wearable equipment, packaging the data and sending the data to the data preprocessing unit, preprocessing the data by the data preprocessing unit according to different data, finally transmitting the data to the data normalizing unit, uniformly normalizing the data and transmitting the normalized data to the feature extraction and early warning module based on transpose coding.
According to the invention, the feature extraction and early warning module based on transpose coding comprises an environment information processing unit, a physiological information processing unit and a positioning information processing unit which are connected in parallel; the environment information processing unit is connected with the environment information early warning unit in series; the physiological information processing unit is further connected with the physiological information early warning unit and the physiological information identification unit in series and parallel; the positioning information processing unit is further connected with the positioning information decoding unit, the positioning information fusion unit and the early warning unit beyond the safety zone in series;
the environment information processing unit is used for processing environment gas information and environment voltage information and adjusting an environment gas concentration early warning value and an environment electric field voltage early warning value;
The environment information early warning unit is used for comparing the environment information with a preset early warning value and giving an alarm when the environment information exceeds the early warning value;
the physiological information processing unit is used for processing body temperature, heart rate, blood oxygen saturation, X, Y, Z triaxial acceleration data and X, Y, Z triaxial angular velocity data, and setting personalized data early warning threshold values according to different individual conditions;
the physiological information early warning unit is used for judging a reasonable numerical range according to different individuals and outputting early warning information; the physiological information early warning refers to analyzing a result obtained after processing a received physiological signal, and immediately alarming after identifying abnormality; abnormality means that data outside the normal range of physiological medicine is found by analyzing information of heart rate, blood oxygen, and body temperature of the wearer in combination with physical signs of the wearer;
the physiological information identification unit realizes behavior classification and abnormal behavior early warning through a multichannel attention network model based on the extrusion excitation module and the transposition coding module; wherein the behavior classification refers to human behavior recognition (Human Activity Recognition, HAR);
the positioning information processing unit analyzes and processes the collected LBS and GPS positioning signals to realize the analysis of the LBS original base station data, and synthesizes the data information of the GPS and the LBS to obtain data positioning;
The positioning information decoding unit decodes LBS data to obtain visible data information; the positioning information fusion unit fuses the collected data from different sources;
and after the positioning is confirmed, the early warning unit exceeding the safety zone is compared with a safety zone defined in advance, and early warning is carried out when the safety zone exceeds the safety zone range.
According to the invention, the multichannel attention network model based on the extrusion excitation module and the transposition coding module comprises a multichannel characteristic and abnormality monitoring module, a transposition coding module and an output module which are connected in series;
the multi-channel characteristic and abnormality monitoring module comprises a noise superposition module, a data reconstruction module, an abnormality monitoring module, a characteristic vector and abnormality information output module which are connected in series; the noise superposition module is used for superposing Gaussian noise on a multi-channel data random selection channel, inputting the multi-channel data random selection channel into the data reconstruction module, and inputting data containing noise into the data reconstruction module;
the data reconstruction module is used for extracting higher-layer expression of an input signal; the data reconstruction module comprises an input layer, a hidden layer and an output layer, wherein the input layer is decoded to the hidden layer, and the hidden layer is encoded to the output layer; the output of the hidden layer is used as the characteristic expression of the original input data, the characteristic vector of the hidden layer is input to the characteristic vector and abnormal information output module, the output layer is used as the input reconstruction data, and the reconstruction data is output to the abnormal monitoring module;
The abnormality monitoring module is used for carrying out abnormality monitoring on input data; inputting the screened abnormal data into a feature vector and abnormal information output module;
the characteristic vector and anomaly information output module acquires the characteristic vector from the data reconstruction module, acquires the anomaly data from the anomaly monitoring module, synthesizes the two data, eliminates the characteristic vector which is input and detected as anomaly, and finally outputs all the data to the transposition coding module in three ways;
according to the invention, the transpose encoding module comprises a transpose module, an SE module, a layer normalization module and an output module which are connected in series;
the transposition module transposes the blind axis of the input data, so that different channel weights are redistributed, different information of three dimensions is read, and then the transposed three channel information is input into the SE module;
the SE module is an extrusion and excitation network module;
the layer normalization module performs batch layer normalization on the output of the SE module and inputs the output data into the output module;
the output module inputs the output of the SE module to the full-connection layer, then inputs the output to the Softmax classifier, performs classification and identification of behavior information, obtains the probability of each behavior, and obtains the behavior corresponding to the maximum probability, namely the final behavior identification result of the multichannel attention network model based on the extrusion excitation module and the transposed encoding module.
According to the invention, the data display and correction module comprises an information synthesis unit, wherein the information synthesis unit is connected with a parallel physiological information display unit and an early warning information alarm unit in series, then is connected with a data real-time display unit and a user operation unit in series, and then is connected with an error information correction unit and an instruction issuing unit in parallel in series;
the information synthesis unit is used for synthesizing all environmental information, positioning information, physiological information and early warning information;
the physiological information display unit is used for displaying physiological information in real time through a webpage;
the early warning information alarm unit is used for informing an administrator and electric power operators of early warning information through the cloud server;
the data real-time display unit is used for displaying all environment alarm information, positioning early warning information exceeding a safety zone and human behavior identification information in real time through a webpage end;
the user operation unit is used for receiving the operations of a user and an administrator, and comprises database calling, database modification and data alarm threshold modification;
the instruction information issuing unit is used for transmitting and controlling the instructions of the administrator to the wearable equipment through the network.
The wearable remote real-time monitoring method for the electric power operators based on the multimode data is realized by the wearable remote real-time monitoring system for the electric power operators based on the multimode data, and comprises the following steps:
Step S1: management personnel select a monitoring scenario
An administrator sends a command to the wearable equipment through a webpage end, controls the wearable equipment, and selects different collected information and frequencies under different use scenes;
step S2: the sensor collects multi-mode information
Receiving an administrator command from the step S1 on the wearable equipment, and controlling a sensor to selectively acquire physiological information, environmental information and position information of a user, wherein the physiological information comprises acceleration data of X, Y, Z three axes acquired by an acceleration sensor, X, Y, Z three-axis angular velocity data acquired by an angular velocity sensor, heart rate and body temperature, the environmental information comprises near electricity data and dangerous gas data, and the position information comprises GPS (global positioning system), LBS (location based service) data, media access control addresses and signal strength;
step S3: uploading cloud server
The multi-mode information collected by the wearable equipment is transmitted to a cloud server in real time through a TCP/IP protocol;
step S4: judging data type
Classifying the multi-mode information stored by the cloud server, wherein the concentration of ambient gas and the voltage of an electric field are environmental data, the GPS, LBS, media access control address and signal intensity data are positioning data, and the acceleration, angular velocity, temperature and heart rate data are behavior data;
Step S5: physiological information preprocessing
The pretreatment of physiological information specifically means: sequentially performing filtering, normalization, fusion and time sequence-based interception;
step S6: environmental information early warning
Comparing the pre-warning value with the ambient gas concentration value acquired by the current ambient gas sensor according to the pre-warning value set by a user, and judging whether the ambient gas concentration value is out of range; when the received value is larger than a preset early warning value, alarm processing is carried out;
step S7: building and training behavior recognition model
The behavior recognition model is a multichannel attention model network based on transposed encoding, and comprises a multichannel feature extraction and anomaly monitoring module and a transposed encoding module;
the parameters set by the user include: the shape of the input data, the shape of the remodeled data and the superimposed noise parameters are subjected to iterative training, parameters of the behavior recognition model are continuously optimized, and finally the trained behavior recognition model is obtained, wherein the trained behavior recognition model is shown as a formula (I):
in the formula (I), A is the number of samples, M is the number of categories, y ij As a sign function, if the true class of sample i is equal to j, 1 is taken, noThen take 0, p ij The prediction probability of the observation sample i belonging to the category j;
step S8: real-time behavior recognition
Inputting the physiological information which is acquired in real time and subjected to pretreatment into a trained multichannel attention model network behavior recognition model based on transposition codes to perform real-time recognition of current behaviors, and outputting a classification result corresponding to the physiological information;
step S9: positioning data analysis
The GPS signal and the LBS signal are processed separately, and step S9 includes the sub-steps of:
step S91: data decoding
Decoding the LBS signal, wherein the LBS signal comprises a communication base station code, and inquiring position information corresponding to the code in an open source data warehouse to obtain decoded base station position information;
step S93: multimode data fusion positioning
Performing fusion positioning according to the analyzed LBS data and GPS data; accurate positioning information is obtained;
step S10: real-time information display
And displaying the behavior recognition result output by the multichannel attention model network based on transposed coding through a behavior display unit in the data display and correction module.
According to the preferred embodiment of the present invention, step S7 sets parameters x, y, z of the input data, which are the length, width, and channel number of the original input data, respectively;
setting a noise parameter L of a noise superposition module, wherein L is a random 0 value or Gaussian noise is adopted for superposition;
Setting initial parameters w and b of a data reconstruction module, wherein w is a network weight, and b is a bias coefficient;
setting parameters x ', y ', z ', and respectively realizing the length, width and channel number of the original input data after feature extraction by a data reconstruction module;
setting batch_Size and Window_Size, wherein batch_Size refers to the number of each Batch of behavior samples of the multi-channel feature extraction and anomaly monitoring module, and Window_Size refers to the length of sample data;
the input layer of the data reconstruction module in the multi-channel feature extraction and anomaly monitoring module is provided with c 1 Neurons, hidden layer c 2 The output layer of each neuron is c 3 A neuron;
setting the input and output comparison function in the abnormality monitoring module as f, and judging the threshold value as alpha;
setting the input shape of the SE module as x ', y ', z ' which are the output shape parameters of the data reconstruction module respectively, wherein the scaling parameters of the accounting part are SERADIO;
step S71: raw data input
Assuming that the input data is formed by fusion of measured input data of n triaxial acceleration sensors and triaxial angular velocity sensors, the Size is physiological information of batch_Size×Window_Size×6n, window_Size is the data length, and 6n is the number of data channels;
Step S72: data input multichannel feature extraction and anomaly monitoring module
The physiological information with the Size of batch_Size×Window_Size×6n is sent to a multi-channel feature extraction and anomaly monitoring module, the physiological information is split into x, y, … and z-channel data, the x, y, … and z-channel data are input to a noise superposition module, the noise superposition module randomly selects channels of the multi-channel data to superpose Gaussian noise, and the channels are input to a data reconstruction module;
the data reconstruction module is a three-layer network model and comprises an input layer, a hidden layer and an output layer, wherein the number of neurons of the input layer is c 1 The number of neurons in the hidden layer is c 2 The output layer has c 3 Neurons, c 1 、c 2 And c 3 The relationship of (a) is formula (II):
c 1 >c 2 ≥c 3 (II)
final output size of
Inputting the characteristic information of the device into an abnormality monitoring module; the anomaly monitoring module multiplexes the network structure of the data reconstruction module, and outputs layer data and noise by comparingThe data output by the module are overlapped to obtain a reconstruction error, an anomaly score obtained through the reconstruction error is calculated, the anomaly score is compared with a judgment threshold alpha, and the abnormal behavior is obtained when the anomaly score exceeds the judgment threshold alpha; the loss function of the reconstruction error is of formula (III):
L(x,z)＝‖x-z‖ 2 (III)
in the formula (III), x is original input layer data before noise superposition, and z is output layer data;
Step S73: data input transposition coding module
Will be of the size of
The feature vector of (a) is input into a behavior recognition module; setting the original input shape as xxyxz, respectively representing the width, height and channel number, and transposing the original input along different axes to obtain data of yxz x and zxx x y; so as to redistribute different channel weights, read three pieces of information with different dimensions, and then input the transposed and transposed three pieces of channel information into the SE module;
step S74: data input SE module
The SE module is an extrusion and excitation network module, and firstly calculates the channel attention in an effective mode by utilizing global average set characteristics, wherein the channel attention comprises a pooling layer, a full-connection layer, a convolution layer, a full-connection layer and a convolution layer which are sequentially linked;
wherein the attention mechanism combining the average pooling operation and the maximum pooling operation is described as formula (IV):
Wc＝σ(f{w 1,2 }(AP(χ))+f{w 1 ,w 2 }(MP(χ))) (IV)
in the formula (IV), χ is tensor data input to three SE modules,
representing average pooling and maximum pooling operations, respectively; sigma is a sigmod function, w 1 ,w 2 Is a network parameter used for adjusting a scale factor; formula (IV) is described as formula (V):
Wc＝σ(w 2 ReLU(w 1 AP(χ))+w 2 ReLU(w 1 MP(χ)))(V)
step S75: addition polymerization
The three attention channels output by the SE module are aggregated by addition, then Z_Pooling is used to preserve the feature representation, while depth is compressed, and the connected and aggregated feature information is represented as formula (VI):
Z_Pooling(χ)＝[MaxPool 0d (χ),AvgPool 0d (χ)](VI)
In formula (VI), 0d is the data of the 0 th dimension where maximum pooling and average pooling occur;
step S76: layer normalization
Carrying out batch layer normalization on the aggregated output to obtain 1 XyXz output;
step S77: outputting the unit identification result
The physiological information is converted into scalar quantity from vector through the unfolding layer, and the Size of the information Output after passing through the full-connection layer is batch_size multiplied by output_length; classifying and identifying physiological information through a Softmax classifier; inputting physiological information characteristics with information size into a Softmax classifier, solving the probability of each behavior, and obtaining the behavior corresponding to the maximum probability, namely the final behavior recognition result of the multi-channel attention model network based on transposed encoding.
According to a preferred embodiment of the invention, step S5 comprises the steps of:
step S51: physiological information denoising
Denoising the physiological information by a wavelet threshold method;
step S52: physiological information normalization
Carrying out mean normalization processing by a mean normalization method to enable the characteristics of different dimensions to be in the same numerical magnitude;
step S53: multimodal data fusion
Aligning the acquired multiple physiological information according to the time stamp;
step S54: time series based interception
And referring to parameters preset by a user, including the size of a sliding window, the cutting frequency and the step length of the sliding window, performing sliding window processing on the physiological information processed in the step S53, so that the physiological information is input into a multichannel attention model network based on transposed encoding in the form of information blocks.
The beneficial effects of the invention are as follows:
1. the invention provides a multi-mode data-based wearable remote real-time monitoring system for electric power operators, which is used for processing and analyzing various data collected by wearable equipment to realize monitoring of the electric power operators.
2. The invention provides a hierarchical structure of the wearable equipment, the cloud platform and the webpage end, and each layer has definite division of labor and close matching, so that various real-time information of workers can be clearly mastered and the abnormality of the workers can be treated.
3. In the special application scene of the electric power operation, special data are collected, and a special algorithm is designed. Aiming at the problems frequently encountered by electric workers, near-electricity induction information and harmful gas information are collected, and a special indoor and outdoor positioning algorithm and a human activity recognition algorithm are designed. The monitoring system has more pertinence in the power industry, can adapt to industry requirements, and has higher accuracy and use value.
Drawings
FIG. 1 is a schematic diagram of the module composition and connection relationship of the multi-mode data based power worker wearable remote real-time monitoring system of the invention.
Fig. 2 is a flow chart of the method for wearable remote real-time monitoring of electric power operators based on multimode data.
Fig. 3 is a schematic diagram of a classification and identification network based on a feature extraction and early warning module of transposed encoding.
FIG. 4 is a schematic diagram of a multi-channel feature extraction and anomaly monitoring module according to the present invention.
FIG. 5 is a schematic diagram of a transposed encoding module in accordance with the present invention.
FIG. 6 is a graph of triaxial acceleration data employed in the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the examples and the accompanying drawings, and it is apparent that the specific examples described herein are only for explaining the present invention and are not intended to limit the present invention.
Example 1
In the management of power system operators, a plurality of potential safety hazards exist, common dangers are that the operators enter a high-voltage electric field by mistake, touch the high-voltage electric field by mistake, are exposed to toxic and harmful gases or fall down in operation, and the like, and various potential safety hazards faced by the operators are difficult to discover and report in time, so that the operators cannot be treated in time, and the personal safety of the operators cannot be effectively guaranteed. Based on the problems, a set of remote monitoring system based on the wearable equipment has better practical effect. The electric power operation personnel can choose to wear an intelligent bracelet, an inner heart rate acquisition module of the bracelet, a triaxial acceleration sensor, a triaxial angular velocity sensor, a GPS and LBS positioning module are used for acquiring physiological information and positioning information, and meanwhile, the portable environment information acquisition module is carried, so that the content of various toxic and harmful gases in the air can be acquired, and multi-gear high-voltage electricity monitoring can be carried out. When the wearer falls, the data of the x, y and z three-axis acceleration sensors acquired by the intelligent bracelet are shown in fig. 6. The intelligent bracelet and the environment information acquisition module can acquire and transmit data to the cloud server in real time, and the cloud server stores and calculates the data to obtain corresponding identification results and early warning information. The power system operator management application platform can display the calculation result of the cloud platform in real time and feed back the calculation result to the manager, so that the manager can conveniently and timely process emergency.
The system comprises an operator real-time information acquisition module, a multi-mode data preprocessing module, a feature extraction and early warning module based on transposition codes and a data display and correction module which are sequentially connected, as shown in fig. 1;
the real-time information acquisition module of the operating personnel is used for: collecting real-time information through the wearable equipment, wherein the real-time information comprises environment information, physiological information and positioning information;
the multi-mode data preprocessing module is used for: receiving real-time information from the wearable device and preprocessing the real-time information;
the feature extraction and early warning module based on transpose coding is used for: processing environment, physiological and positioning information data, and performing human behavior recognition through a feature extraction algorithm based on transposed encoding;
the data display and correction module is used for: receiving data from a data identification and early warning module, synthesizing various data, and performing visual display and alarm on a webpage end; meanwhile, an administrator operation interface is provided at the webpage end for an administrator to send instructions and correction data.
Example 2
The wearable remote real-time monitoring system for electric power operators based on multimode data according to the embodiment 1 is characterized in that:
The real-time information acquisition module of the operator comprises an environment information acquisition unit, a physiological information acquisition unit, a positioning information acquisition unit and a data transmission unit which are connected in series;
the environment information acquisition unit is used for: acquiring environmental information; the physiological information acquisition unit is used for: acquiring physiological information; the positioning information acquisition unit is used for: acquiring positioning information; the data transmission unit is used for: and remotely transmitting the collected behavior data of the user to a database of the cloud server in real time through a TCP/IP protocol.
The environment information acquisition unit comprises an air quality detection unit and a near electricity induction unit;
the air quality detection unit is used for: reading environmental data in the air, including the concentration of a plurality of common toxic and harmful gases such as sulfur dioxide, carbon monoxide and methane;
the near electricity induction unit is used for: the electric field intensity near the human body is read, and power plant monitoring and alarming for nearby 220V, 10kV and 35KV voltages are sensitively and reliably realized;
the air quality detection system takes a sensor module as a starting point, the sensor module collects gas data in the air, and then the collected data are transmitted to the STM32 through 485 communication. STM32 chip is the core of whole air quality detecting system, and the air data that will pass back is handled. The near-electric induction system surrounds an open-source JW5808 chip, a multi-gear near-electric induction circuit controlled by a sliding switch is built, and electric field monitoring and alarming of nearby 220V, 10kV and 35KV voltages can be flexibly and reliably achieved.
The physiological information acquisition unit comprises an acceleration sensor, an angular velocity sensor, a temperature sensor and a PPG sensor;
the acceleration sensor is used for acquiring acceleration data of X, Y, Z triaxial; the angular velocity sensor is used for acquiring X, Y, Z triaxial angular velocity data of the angular velocity sensor; the temperature sensor is used for acquiring body temperature data; the PPG sensor is used for measuring heart rate and blood oxygen saturation;
the positioning information acquisition unit comprises a global positioning system (Global Positioning System, GPS) and a Wi-Fi base station information receiving system;
the global positioning system is used for acquiring GPS signals and service signals LBS which are spread around geographic position data; the Wi-Fi base station information receiving system is configured to obtain a media access control address (Media Access Control Address, MAC) and a signal strength.
The pretreatment comprises the following steps: and denoising filtering, normalization, data fusion and time sequence based interception are sequentially carried out. Thus, smooth, diversified and unified multi-modal data are obtained, and then the intercepted behavior data are input into a feature extraction and early warning module based on transposed encoding.
The normalization means: carrying out mean value normalization processing on the numerical value data in the behavior information, namely converting the numerical values of all the similar sensors into values between [ -1,1 ];
The data fusion refers to: aligning behavior information carried by different sensors according to time stamps, and fusing;
the time sequence-based interception refers to: the continuous time sequence acquired by the sensor is segmented by a sliding window method, so that a complete action data is ensured to fall in a sliding window.
The multi-mode data preprocessing module comprises a data receiving unit, a data preprocessing unit and a data normalization unit which are connected in sequence;
the data receiving unit is used for receiving various data from the wearable equipment, packaging the data and sending the data to the data preprocessing unit, preprocessing the data by the data preprocessing unit according to different data, finally transmitting the data to the data normalization unit, uniformly normalizing the data and transmitting the normalized data to the feature extraction and early warning module based on transpose coding.
The data receiving unit receives the data acquired by the real-time information acquisition module of the operator through a TCP/IP protocol and stores the data into the cloud server database; the data preprocessing unit performs different data preprocessing algorithms for different data; the data normalization unit performs mean normalization on data of different dimensions, namely, values of all the same type of sensors are transformed to be between [ -1,1], and then the processed information is transmitted to the data display and correction module.
The feature extraction and early warning module based on transpose coding comprises an environment information processing unit, a physiological information processing unit and a positioning information processing unit which are connected in parallel; the environment information processing unit is connected with the environment information early warning unit in series; the physiological information processing unit is further connected with the physiological information early warning unit and the physiological information identification unit in series and parallel; the positioning information processing unit is further connected with the positioning information decoding unit, the positioning information fusion unit and the early warning unit beyond the safety zone in series;
the environment information processing unit is used for processing the environment gas information and the environment voltage information, considering the actual working environment of workers, and adjusting the environment gas concentration early-warning value and the environment electric field voltage early-warning value according to the actual situation;
the environment information early warning unit is used for comparing the environment information with a preset early warning value and giving an alarm when the environment information exceeds the early warning value;
the physiological information processing unit is used for processing the body temperature, the heart rate, the blood oxygen saturation, the acceleration data of X, Y, Z triaxial and the angular velocity data of X, Y, Z triaxial, and setting personalized data early warning threshold values according to different individual conditions;
the physiological information early warning unit is used for judging a reasonable numerical range according to different individuals and outputting early warning information; the physiological information early warning refers to analyzing a result obtained after processing a received physiological signal, and immediately alarming after identifying abnormality; abnormality means that data outside the normal range of physiological medicine is found by analyzing information of heart rate, blood oxygen, and body temperature of the wearer in combination with physical signs of the wearer;
The physiological information identification unit realizes behavior classification and abnormal behavior early warning through a multichannel attention network model based on the extrusion excitation module and the transposition coding module; wherein the behavior classification refers to human behavior recognition (Human Activity Recognition, HAR); the activity of the worker can be confirmed through behavior recognition.
The positioning information processing unit analyzes and processes the collected LBS and GPS positioning signals to realize the analysis of the LBS original base station data, and synthesizes the data information of the GPS and the LBS to obtain reasonable data positioning;
the positioning information decoding unit decodes the LBS data to obtain visible data information; the LBS base station positioning is based on a positioning mode of a signal tower of a communication operator, longitude and latitude information of a mobile phone SIM card is obtained through the signal tower, the position point is displayed on a map through calculation and butting with an electronic map API, and the purpose of positioning is achieved. Meanwhile, the LBS base station positioning has mandatory, and the base station positioning server end actively initiates positioning to the terminal, thereby playing an important role in supervision service;
the positioning information fusion unit fuses the collected data from different sources; the positioning is more accurate; the location mode of LBS base station location belongs to earth surface location, all signals are from the cell phone signal tower, as long as the cell phone can implement location in the place where the signals are received, the base station location signals have extremely strong penetrability in ground buildings and tunnels, and the defect of satellite location is overcome. Secondly, LBS has mandatory, when the terminal does not request the position, active positioning can be carried out, and the positioning of the LBS base station has stronger supervision capability; the GPS positioning has better precision, and the positioning is more accurate through fusion positioning;
And after confirming positioning, the early warning unit exceeding the safety zone compares the safety zone with a pre-defined safety zone, and early warning is carried out when the safety zone exceeds the safety zone.
The multichannel attention network model based on the extrusion excitation module and the transposition coding module comprises a multichannel characteristic and anomaly monitoring module, a transposition coding module and an output module which are connected in series;
the multi-channel characteristic and abnormality monitoring module comprises a noise superposition module, a data reconstruction module, an abnormality monitoring module, a characteristic vector and abnormality information output module which are connected in series; the noise superposition module is used for superposing Gaussian noise on a multi-channel data random selection channel and inputting the multi-channel data random selection channel into the data reconstruction module, the traditional non-superposition noise method is simply used for minimizing reconstruction errors, the learned characteristics are likely to be only replication of original input, and in order to avoid the problem, the system provided by the invention introduces a noise injection strategy, reduces information contained in the original input, and data containing noise is then input into the data reconstruction module;
the data reconstruction module is used for extracting higher-level expression of the input signal; the network parameters are adjusted through the error between the input and the reconstructed signals, the lost information is filled through learning trial, and the input data structure is further learned, so that the extracted characteristics can reflect the characteristics of the original input more, namely, the aim of network training is to make the input and the output the same as possible, and the method belongs to non-supervision learning; the data reconstruction module comprises an input layer, a hidden layer and an output layer, wherein the input layer is decoded to the hidden layer, and the hidden layer is encoded to the output layer; the output of the hidden layer is used as the characteristic expression of the original input data, the characteristic vector of the hidden layer is input to the characteristic vector and abnormal information output module, the output layer is used as the input reconstruction data, and the reconstruction data is output to the abnormal monitoring module;
The abnormality monitoring module is used for carrying out abnormality monitoring on input data; the training network multiplexes the three-layer network used by the data reconstruction module, namely: an input layer, a hidden layer, and an output layer; the network training method is that firstly, normal data are used for training, low latitude representation of an original data set is learned, and the original data set is restored to a certain extent; when the test set is used, the network structure can perform encoding and decoding according to the format of normal data, the errors of the input layer and the output layer are in a certain range, which is called error distribution, and when abnormal data is input, the errors of the output layer and the input layer exceed the error distribution range, and the abnormal data can be considered if the errors of one data after encoding and decoding exceed the error distribution range. The module inputs the screened abnormal data into the characteristic vector and abnormal information output module;
the feature vector and anomaly information output module acquires feature vectors from the data reconstruction module, acquires anomaly data from the anomaly monitoring module, synthesizes the two data, eliminates the feature vectors which are input and detected as anomalies, and finally outputs all the data to the transposed encoding module in three ways;
The transpose coding module comprises a transpose module, an SE module, a layer normalization module and an output module which are connected in series;
the transposition module transposes the blind axis of the input data, so that different channel weights are redistributed, different information of three dimensions is read, and then the transposed three channel information is input into the SE module;
the SE module is an extrusion and excitation network module; the method has high flexibility in that the method can be directly applied to the existing network structure, is proposed by Jiehu et al in the text of Squeeze-and-Excitation Networks published in 2019, comprises a pooling layer, a full-connection layer, a convolution layer, a full-connection layer and a convolution layer which are sequentially linked, and calculates the channel attention in an effective mode by utilizing the global average set characteristic;
the layer normalization module performs batch layer normalization on the output of the SE module, and inputs the output data into the output module;
the output module inputs the output of the SE module to the full-connection layer, then inputs the output to the Softmax classifier, carries out classification and identification of behavior information, obtains the probability of each behavior, and obtains the behavior corresponding to the maximum probability, namely the final behavior identification result of the multichannel attention network model based on the extrusion excitation module and the transposition coding module.
The data display and correction module comprises an information comprehensive unit, wherein the information comprehensive unit is connected with a physiological information display unit and an early warning information alarm unit in parallel in series, then is connected with a data real-time display unit and a user operation unit in series, and then is connected with an error information correction unit and an instruction issuing unit in parallel in series;
the information synthesis unit is used for synthesizing all environmental information, positioning information, physiological information and early warning information;
the physiological information display unit is used for displaying physiological information in real time through a webpage;
the early warning information alarming unit is used for informing an administrator and electric power operation personnel of early warning information through a short message reminding mode and the like by the cloud server;
the data real-time display unit is used for displaying all environment alarm information, positioning early warning information exceeding a safety zone and human behavior identification information in real time through a webpage end;
the user operation unit is used for receiving the operations of a user and an administrator, and comprises database calling, database modification and data alarm threshold modification;
the instruction information issuing unit is used for transmitting and controlling the instructions of the administrator to the wearable equipment through the network.
Example 3
The method for monitoring the power operation staff in a wearable remote real-time manner based on the multimode data is realized by the system for monitoring the power operation staff in a wearable remote real-time manner based on the multimode data in embodiment 1 or 2, as shown in fig. 2, and comprises the following steps:
Step S1: management personnel select a monitoring scenario
An administrator sends a command to the wearable equipment through a webpage end, controls the wearable equipment, and selects different collected information and frequencies under different use scenes;
step S2: the sensor collects multi-mode information
Receiving an administrator command from the step S1 on the wearable equipment, and controlling a sensor to selectively acquire physiological information, environmental information and position information of a user, wherein the physiological information comprises acceleration data of X, Y, Z three axes acquired by an acceleration sensor, X, Y, Z three-axis angular velocity data acquired by an angular velocity sensor, heart rate and body temperature, the environmental information comprises near electricity data and dangerous gas data, and the position information comprises GPS (global positioning system), LBS (location based service) data, media access control addresses and signal strength;
step S3: uploading cloud server
The multi-mode information collected by the wearable equipment is transmitted to a cloud server in real time through a TCP/IP protocol;
step S4: judging data type
Classifying the multi-mode information stored by the cloud server, wherein the concentration of ambient gas and the voltage of an electric field are environmental data, the GPS, LBS, media access control address and signal intensity data are positioning data, and the acceleration, angular velocity, temperature and heart rate data are behavior data; so as to process and analyze the different types of data respectively.
Step S5: physiological information preprocessing
The pretreatment of physiological information specifically means: sequentially performing filtering, normalization, fusion and time sequence-based interception;
step S5, comprising the steps of:
step S51: physiological information denoising
Denoising the physiological information by a wavelet threshold method;
step S52: physiological information normalization
Carrying out mean normalization processing by a mean normalization method to enable the characteristics of different dimensions to be in the same numerical magnitude;
step S53: multimodal data fusion
Aligning a plurality of physiological information acquired by the sensor according to the time stamp;
step S54: time series based interception
Referring to parameters preset by a user, including the size of a sliding window, the cutting frequency and the step length of the sliding window, sliding window processing is carried out on the physiological information processed in the step S53, so that the physiological information is input into a multichannel attention model network based on transposed encoding in the form of information blocks;
in step S51, the physiological information is denoised by a wavelet thresholding method, specifically:
the actual measurement signal is assumed to be: f (t) =s (t) +e (t), t=1, 2, …, N, s (t) is the original signal, f (t) is the noise-containing signal, e (t) is white gaussian noise, e (t) to N (0, σ) 2 ) Sigma is the noise intensity, and the denoising process removes noise e (t) from the signal f (t) to obtain the best approximation of the original signal s (t);
firstly, performing discrete sampling to obtain an N 'point discrete signal f (x), wherein x=0, 1,2, …, N' -1, and the wavelet transform coefficient is shown in formula (1):
wherein W is f (j, k) is a wavelet coefficient, ψ (2 -j x-k) is a scale function, j is a scale parameter, k is the unit number of the scale function translation, and the recursive implementation method of the formula (1) is obtained through a double-scale equation (2) and a formula (3):
S f (j+1,k)＝S f (j,k)*h(j,k) (2)
W f (j+1,k)＝S f (j,k)*g(j,k) (3)
wherein the symbol "×" represents convolution, h and g represent low-pass and high-pass filters, respectively, S f (0, k) represents the original signal f (k), S f (j, k) represents the approximation coefficient on the j scale, then the wavelet transform reconstruction formula is shown as formula (4):
S f (j-1,k)＝S f (j,k)*h(j,k)+W f (j,k)*g(j,k) (4)
secondly, a general threshold rule is adopted to determine a threshold, and a threshold algorithm formula is shown in a formula (5):
wherein σ=mad/0.6755, MAD is the intermediate value of the absolute value of the first layer wavelet decomposition coefficient, 0.6755 is the adjustment coefficient of gaussian noise standard deviation, and L is the size or length of the signal; then, carrying out nonlinear threshold processing on wavelet transformation coefficients of the measurement signals, processing high-frequency coefficients of each of the 1 st to the V th layers by adopting a soft threshold function, comparing an absolute value of the signals with a threshold value, setting a point smaller than the threshold value as zero, contracting the point larger than or equal to the threshold value towards zero, changing the point value into the difference between the point value and the threshold value, and not processing the low-frequency coefficient of each layer; the soft threshold function is shown in equation (6):
Wherein W is j,k Is W f (j, k) shorthand, T is a threshold parameter greater than zero.
And finally, carrying out wavelet reconstruction on the signals according to a formula (3) according to the low-frequency coefficient of the N layer of wavelet decomposition and the high-frequency coefficients of the 1 st layer to the V layer after quantization treatment, and obtaining the denoised signals.
Step S6: environmental information early warning
According to the pre-warning value set by a user in advance, the default values are that the air concentration of carbon monoxide is not more than 0.08%, the sulfur dioxide concentration is not more than 0.0005%, the carbon dioxide concentration is not more than 1.5% and the methane concentration is not more than 0.75%, comparing with the ambient gas concentration value acquired by the current ambient gas sensor, and judging whether the ambient gas is out of range; when the received value is larger than a preset early warning value, alarm processing is carried out;
step S7: building and training behavior recognition model
The behavior recognition model is a multichannel attention model network based on transposed encoding, and comprises a multichannel feature extraction and anomaly monitoring module and a transposed encoding module; as shown in fig. 3, 4 and 5;
the parameters set by the user include: the shape of input data, the shape of the remodeled data (length, width and channel number after feature extraction) and the superimposed noise parameters are subjected to iterative training, parameters of a behavior recognition model are continuously optimized, and finally the trained behavior recognition model is obtained, wherein the trained behavior recognition model is shown as a formula (I):
In the formula (I), A is the number of samples, M is the number of categories, y ij As a sign function (0 or 1), 1 is taken if the true class of sample i is equal to j, otherwise 0, p is taken ij The prediction probability of the observation sample i belonging to the category j;
step S7, setting parameters x, y and z of input data, wherein the parameters x, y and z are the length, width and channel number of original input data respectively;
setting a noise parameter L of a noise superposition module, wherein L is a random 0 value or Gaussian noise is adopted for superposition;
setting initial parameters w and b of a data reconstruction module, wherein w is a network weight, and b is a bias coefficient;
setting parameters x ', y ', z ', and respectively realizing the length, width and channel number of the original input data after feature extraction by a data reconstruction module;
setting batch_Size and Window_Size, wherein batch_Size refers to the number of each Batch of behavior samples of the multi-channel feature extraction and anomaly monitoring module, and Window_Size refers to the length of sample data;
the input layer of the data reconstruction module in the multi-channel feature extraction and anomaly monitoring module is provided with c 1 Neurons, hidden layer c 2 The output layer of each neuron is c 3 A neuron;
setting the input and output comparison function in the abnormality monitoring module as f, and judging the threshold value as alpha;
setting the input shape of the SE module as x ', y ', z ' which are the output shape parameters of the data reconstruction module respectively, wherein the scaling parameters of the accounting part are SERADIO;
As shown in fig. 3, the specific implementation procedure is as follows:
step S71: raw data input
Assuming that the input data is formed by fusion of measured input data of n triaxial acceleration sensors and triaxial angular velocity sensors, the Size is physiological information of batch_Size×Window_Size×6n, window_Size is the data length, and 6n is the number of data channels;
step S72: data input multichannel feature extraction and anomaly monitoring module
The physiological information with the Size of batch_Size×Window_Size×6n is sent to a multi-channel feature extraction and abnormality monitoring module, as shown in fig. 4, the physiological information is split into x, y, … and z channel data, the x, y, … and z channel data are input into a noise superposition module, the noise superposition module randomly selects channels for superposition of Gaussian noise on the multi-channel data, and the data reconstruction module is input;
the data reconstruction module is a three-layer network model and comprises an input layer, a hidden layer and an output layer, wherein the number of neurons of the input layer is c 1 The number of neurons in the hidden layer is c 2 The output layer has c 3 Neurons, c 1 、c 2 And c 3 The relationship of (a) is formula (II):
c 1 >c 2 ≥c 3 (II)
final output size of
Inputting the characteristic information of the device into an abnormality monitoring module; the anomaly monitoring module multiplexes the network structure of the data reconstruction module, obtains reconstruction errors by comparing the data of the output layer with the data output by the noise superposition module, calculates anomaly scores obtained by the reconstruction errors, compares the anomaly scores with a judgment threshold alpha, and is abnormal when the anomaly scores exceed the judgment threshold alpha; the loss function of the reconstruction error is of formula (III):
L(x,z)＝ ‖x-z‖ 2 (III)
In the formula (III), x is original input layer data before noise superposition, and z is output layer data;
step S73: data input transposition coding module
Will be of the size of
The feature vector of (a) is input into a behavior recognition module; setting the original input shape as xxyxz, respectively representing the widthThe number of high channels and the number of channels are respectively transposed along different axes to obtain the data of x y x z and z x y x z; so as to redistribute different channel weights, read three pieces of information with different dimensions, and then input the transposed and transposed three pieces of channel information into the SE module; the transposed encoding module based on three-channel attention is capable of capturing interaction information between spatial dimensions of input sensor tensors, time, dimensions of multiple sensor modalities, and the like.
Step S74: data input SE module
The SE module is an extrusion and excitation network module, and firstly calculates the channel attention in an effective mode by utilizing global average set characteristics, wherein the channel attention comprises a pooling layer, a full-connection layer, a convolution layer, a full-connection layer and a convolution layer which are sequentially linked;
wherein the attention mechanism combining the average pooling operation and the maximum pooling operation is described as formula (IV):
Wc＝σ(f{w 1,2 }(AP(χ))+f{w 1 ,w 2 }(MP(χ))) (IV)
in the formula (IV), χ is tensor data input to three SE modules, 
Representing average pooling and maximum pooling operations, respectively; sigma is a sigmod function, w 1 ,w 2 Is a network parameter used for adjusting a scale factor; formula (IV) is described as formula (V):
Wc＝σ(w 2 ReLU(w 1 AP(χ))+w 2 ReLU(w 1 MP(χ))) (V)
the two fully connected layers in the SE module can be described by equation (6), equation (6) uses two linear projections to assign a corresponding weight to each channel.
Step S75: addition polymerization
The three attention channels output by the SE module are aggregated by addition, then Z_Pooling is used to preserve the feature representation, while depth is compressed, and the connected and aggregated feature information is represented as formula (VI):
Z_Pooling(χ)＝[MaxPool 0d (χ),AvgPool 0d (χ)] (VI)
in formula (VI), 0d is the data of the 0 th dimension where maximum pooling and average pooling occur; that is, z_pooling can reduce the 0 th dimension of a given input tensor to 2 dimensions by aggregating two merged features, and a tensor shaped as x y x Z will be converted to a tensor shaped as 2 x y x Z by z_pooling.
Step S76: layer normalization
Carrying out batch layer normalization on the aggregated output to obtain 1 XyXz output;
step S77: outputting the unit identification result
The physiological information is converted into scalar quantity from vector through the unfolding layer, and the Size of the information Output after passing through the full-connection layer is batch_size multiplied by output_length; classifying and identifying physiological information through a Softmax classifier; inputting physiological information characteristics with information size into a Softmax classifier, solving the probability of each behavior, and obtaining the behavior corresponding to the maximum probability, namely the final behavior recognition result of the multi-channel attention model network based on transposed encoding.
Step S8: real-time behavior recognition
Inputting the physiological information which is acquired in real time and subjected to pretreatment into a trained multichannel attention model network behavior recognition model based on transposition codes to perform real-time recognition of current behaviors, and outputting a classification result corresponding to the physiological information; including standing, sitting, lying down, walking, running, climbing stairs, and user-defined specific behavioral activities;
step S9: positioning data analysis
Because the confidentiality of the LBS signal cannot directly acquire the location information, the GPS signal and the LBS signal are processed separately, and step S9 includes the following sub-steps:
step S91: data decoding
Decoding the LBS signal, wherein the LBS signal comprises a communication base station code, and inquiring position information corresponding to the code in an open source data warehouse to obtain decoded base station position information;
step S93: multimode data fusion positioning
Performing fusion positioning according to the analyzed LBS data and GPS data; the LBS base station positioning determines the position of a user by utilizing the measuring and calculating distance of the base station to the distance of the user, the precision is greatly dependent on the distribution and coverage area of the base station, the error can exceed one kilometer, the positioning mode of the LBS base station positioning belongs to the surface positioning, all signals are sourced from a mobile phone signal tower, so long as the mobile phone can implement positioning in places where the signals are received, the base station positioning signals have extremely strong penetrability in ground buildings and tunnels, the defect of GPS satellite positioning is overcome, meanwhile, the LBS base station positioning has mandatory performance, and the base station positioning server end actively initiates positioning to the terminal, thereby better realizing the supervision function;
The GPS positioning uses satellite signals with higher precision, but positioning signals of indoor, underground garages and traffic tunnels can be influenced to increase positioning deviation, and even satellite signals in regions with dense cloud layers can be limited to ensure that the positioning precision is not high or positioning blind areas appear;
the requirement of a specific demand environment cannot be met by the single GPS positioning and the single base station-based positioning, and the positioning blind area is reduced by providing position information by utilizing the base station positioning at the place without GPS signals indoors; meanwhile, in the star searching stage started by the GPS module, the approximate position of a positioning mobile phone is determined through base station positioning, so that the positioning time is shortened, and meanwhile, accurate positioning information is acquired;
step S10: real-time information display
And displaying the behavior recognition result output by the multichannel attention model network based on transposed coding through a behavior display unit in the data display and correction module.
Claims (10)
1. The power operator wearable remote real-time monitoring system based on the multimode data is characterized by comprising an operator real-time information acquisition module, a multimode data preprocessing module, a feature extraction and early warning module based on transposition coding and a data display and correction module which are connected in sequence;
The real-time information acquisition module of the operating personnel is used for: collecting real-time information through the wearable equipment, wherein the real-time information comprises environment information, physiological information and positioning information;
the multi-mode data preprocessing module is used for: receiving real-time information from the wearable device and preprocessing the real-time information;
the feature extraction and early warning module based on transpose coding is used for: processing environment, physiological and positioning information data, and performing human behavior recognition through a feature extraction algorithm based on transposed encoding;
the data display and correction module is used for: receiving data from a data identification and early warning module, synthesizing various data, and performing visual display and alarm on a webpage end; meanwhile, an administrator operation interface is provided at the webpage end for an administrator to send instructions and correction data.
2. The multi-mode data-based power operator wearable remote real-time monitoring system according to claim 1, wherein the operator real-time information acquisition module comprises an environment information acquisition unit, a physiological information acquisition unit, a positioning information acquisition unit and a data transmission unit connected with the environment information acquisition unit, the physiological information acquisition unit and the positioning information acquisition unit in series;
The environment information acquisition unit is used for: acquiring environmental information; the physiological information acquisition unit is used for: acquiring physiological information; the positioning information acquisition unit is used for: acquiring positioning information; the data transmission unit is used for: the collected behavior data of the user is remotely transmitted to a database of the cloud server in real time through a TCP/IP protocol;
further preferably, the environmental information acquisition unit comprises an air quality detection unit and a near electricity induction unit;
the air quality detection unit is used for: reading environmental data in the air, including the concentration of a plurality of toxic and harmful gases;
the near-electricity induction unit is used for: reading the electric field intensity near the human body, and realizing power plant monitoring and alarming;
further preferably, the physiological information acquisition unit comprises an acceleration sensor, an angular velocity sensor, a temperature sensor and a PPG sensor;
the acceleration sensor is used for acquiring acceleration data of X, Y, Z triaxial; the angular velocity sensor is used for acquiring X, Y, Z triaxial angular velocity data of the angular velocity sensor; the temperature sensor is used for acquiring body temperature data; the PPG sensor is used for measuring heart rate and blood oxygen saturation;
further preferably, the positioning information acquisition unit comprises a global positioning system and a Wi-Fi base station information receiving system;
The global positioning system is used for acquiring GPS signals and service signals LBS; the Wi-Fi base station information receiving system is used for acquiring a media access control address and signal strength;
further preferably, the pretreatment comprises: and denoising filtering, normalization, data fusion and time sequence based interception are sequentially carried out.
3. The multi-mode data-based power worker wearable remote real-time monitoring system according to claim 1, wherein the multi-mode data preprocessing module comprises a data receiving unit, a data preprocessing unit and a data normalizing unit which are connected in sequence;
the data receiving unit is used for receiving various data from the wearable equipment, packaging the data and sending the data to the data preprocessing unit, preprocessing the data by the data preprocessing unit according to different data, finally transmitting the data to the data normalizing unit, uniformly normalizing the data and transmitting the normalized data to the feature extraction and early warning module based on transpose coding.
4. The multimode data-based power operator wearable remote real-time monitoring system according to claim 1, wherein the transposed code-based feature extraction and early warning module comprises an environment information processing unit, a physiological information processing unit and a positioning information processing unit which are connected in parallel; the environment information processing unit is connected with the environment information early warning unit in series; the physiological information processing unit is further connected with the physiological information early warning unit and the physiological information identification unit in series and parallel; the positioning information processing unit is further connected with the positioning information decoding unit, the positioning information fusion unit and the early warning unit beyond the safety zone in series;
The environment information processing unit is used for processing environment gas information and environment voltage information and adjusting an environment gas concentration early warning value and an environment electric field voltage early warning value;
the environment information early warning unit is used for comparing the environment information with a preset early warning value and giving an alarm when the environment information exceeds the early warning value;
the physiological information processing unit is used for processing body temperature, heart rate, blood oxygen saturation, X, Y, Z triaxial acceleration data and X, Y, Z triaxial angular velocity data, and setting personalized data early warning threshold values according to different individual conditions;
the physiological information early warning unit is used for judging a reasonable numerical range according to different individuals and outputting early warning information; the physiological information early warning refers to analyzing a result obtained after processing a received physiological signal, and immediately alarming after identifying abnormality; abnormality means that data outside the normal range of physiological medicine is found by analyzing information of heart rate, blood oxygen, and body temperature of the wearer in combination with physical signs of the wearer;
the physiological information identification unit realizes behavior classification and abnormal behavior early warning through a multichannel attention network model based on the extrusion excitation module and the transposition coding module; wherein behavior classification refers to human behavior recognition;
The positioning information processing unit analyzes and processes the collected LBS and GPS positioning signals to realize the analysis of the LBS original base station data, and synthesizes the data information of the GPS and the LBS to obtain data positioning;
the positioning information decoding unit decodes LBS data to obtain visible data information; the positioning information fusion unit fuses the collected data from different sources;
and after the positioning is confirmed, the early warning unit exceeding the safety zone is compared with a safety zone defined in advance, and early warning is carried out when the safety zone exceeds the safety zone range.
5. The multi-mode data-based power operator wearable remote real-time monitoring system according to claim 1, wherein the multi-channel attention network model based on the extrusion excitation module and the transposition coding module comprises a multi-channel characteristic and abnormality monitoring module, a transposition coding module and an output module which are connected in series;
the multi-channel characteristic and abnormality monitoring module comprises a noise superposition module, a data reconstruction module, an abnormality monitoring module, a characteristic vector and abnormality information output module which are connected in series; the noise superposition module is used for superposing Gaussian noise on a multi-channel data random selection channel, inputting the multi-channel data random selection channel into the data reconstruction module, and inputting data containing noise into the data reconstruction module;
The data reconstruction module is used for extracting higher-layer expression of an input signal; the data reconstruction module comprises an input layer, a hidden layer and an output layer, wherein the input layer is decoded to the hidden layer, and the hidden layer is encoded to the output layer; the output of the hidden layer is used as the characteristic expression of the original input data, the characteristic vector of the hidden layer is input to the characteristic vector and abnormal information output module, the output layer is used as the input reconstruction data, and the reconstruction data is output to the abnormal monitoring module;
the abnormality monitoring module is used for carrying out abnormality monitoring on input data; inputting the screened abnormal data into a feature vector and abnormal information output module;
the feature vector and anomaly information output module acquires feature vectors from the data reconstruction module and anomaly data from the anomaly monitoring module, synthesizes the two data, eliminates the feature vectors which are input and detected as anomalies, and finally outputs all the data to the transposed encoding module in three ways.
6. The multimode data-based power operator wearable remote real-time monitoring system of claim 1, wherein the transpose encoding module comprises a transpose module, an SE module, a layer normalization module, and an output module connected in series;
The transposition module transposes the blind axis of the input data, so that different channel weights are redistributed, different information of three dimensions is read, and then the transposed three channel information is input into the SE module;
the SE module is an extrusion and excitation network module;
the layer normalization module performs batch layer normalization on the output of the SE module and inputs the output data into the output module;
the output module inputs the output of the SE module to the full-connection layer, then inputs the output to the Softmax classifier, performs classification and identification of behavior information, obtains the probability of each behavior, and obtains the behavior corresponding to the maximum probability, namely the final behavior identification result of the multichannel attention network model based on the extrusion excitation module and the transposed encoding module.
7. The multi-mode data-based power operator wearable remote real-time monitoring system according to claim 1, wherein the data display and correction module comprises an information synthesis unit, the information synthesis unit is connected with a parallel physiological information display unit and an early warning information alarm unit in series, then is connected with a data real-time display unit and a user operation unit in series, and then is connected with a parallel error information correction unit and an instruction issuing unit in series;
The information synthesis unit is used for synthesizing all environmental information, positioning information, physiological information and early warning information;
the physiological information display unit is used for displaying physiological information in real time through a webpage;
the early warning information alarm unit is used for informing an administrator and electric power operators of early warning information through the cloud server;
the data real-time display unit is used for displaying all environment alarm information, positioning early warning information exceeding a safety zone and human behavior identification information in real time through a webpage end;
the user operation unit is used for receiving the operations of a user and an administrator, and comprises database calling, database modification and data alarm threshold modification;
the instruction information issuing unit is used for transmitting and controlling the instructions of the administrator to the wearable equipment through the network.
8. The method for monitoring the power operation personnel in a wearable remote real-time manner based on the multimode data is realized by the power operation personnel wearable remote real-time monitoring system based on the multimode data according to any one of claims 1 to 7, and is characterized by comprising the following steps:
step S1: management personnel select a monitoring scenario
An administrator sends a command to the wearable equipment through a webpage end, controls the wearable equipment, and selects different collected information and frequencies under different use scenes;
Step S2: the sensor collects multi-mode information
Receiving an administrator command from the step S1 on the wearable equipment, and controlling a sensor to selectively acquire physiological information, environmental information and position information of a user, wherein the physiological information comprises acceleration data of X, Y, Z three axes acquired by an acceleration sensor, X, Y, Z three-axis angular velocity data acquired by an angular velocity sensor, heart rate and body temperature, the environmental information comprises near electricity data and dangerous gas data, and the position information comprises GPS (global positioning system), LBS (location based service) data, media access control addresses and signal strength;
step S3: uploading cloud server
The multi-mode information collected by the wearable equipment is transmitted to a cloud server in real time through a TCP/IP protocol;
step S4: judging data type
Classifying the multi-mode information stored by the cloud server, wherein the concentration of ambient gas and the voltage of an electric field are environmental data, the GPS, LBS, media access control address and signal intensity data are positioning data, and the acceleration, angular velocity, temperature and heart rate data are behavior data;
step S5: physiological information preprocessing
The pretreatment of physiological information specifically means: sequentially performing filtering, normalization, fusion and time sequence-based interception;
Step S6: environmental information early warning
Comparing the pre-warning value with the ambient gas concentration value acquired by the current ambient gas sensor according to the pre-warning value set by a user, and judging whether the ambient gas concentration value is out of range; when the received value is larger than a preset early warning value, alarm processing is carried out;
step S7: building and training behavior recognition model
The behavior recognition model is a multichannel attention model network based on transposed encoding, and comprises a multichannel feature extraction and anomaly monitoring module and a transposed encoding module;
the parameters set by the user include: the shape of the input data, the shape of the remodeled data and the superimposed noise parameters are subjected to iterative training, parameters of the behavior recognition model are continuously optimized, and finally the trained behavior recognition model is obtained, wherein the trained behavior recognition model is shown as a formula (I):
in the formula (I), A is the number of samples, M is the number of categories, y ij As a sign function, if the true class of sample i is equal to j, 1 is taken, otherwise 0, p ij The prediction probability of the observation sample i belonging to the category j;
step S8: real-time behavior recognition
Inputting the physiological information which is acquired in real time and subjected to pretreatment into a trained multichannel attention model network behavior recognition model based on transposition codes to perform real-time recognition of current behaviors, and outputting a classification result corresponding to the physiological information;
Step S9: positioning data analysis
The GPS signal and the LBS signal are processed separately, and step S9 includes the sub-steps of:
step S91: data decoding
Decoding the LBS signal, wherein the LBS signal comprises a communication base station code, and inquiring position information corresponding to the code in an open source data warehouse to obtain decoded base station position information;
step S93: multimode data fusion positioning
Performing fusion positioning according to the analyzed LBS data and GPS data; accurate positioning information is obtained;
step S10: real-time information display
And displaying the behavior recognition result output by the multichannel attention model network based on transposed coding through a behavior display unit in the data display and correction module.
9. The method for monitoring the wearable remote real-time of the electric power operator based on the multimode data according to claim 8, wherein the step S7 is to set parameters x, y and z of the input data, which are respectively the length, the width and the channel number of the original input data;
setting a noise parameter L of a noise superposition module, wherein L is a random 0 value or Gaussian noise is adopted for superposition;
setting initial parameters w and b of a data reconstruction module, wherein w is a network weight, and b is a bias coefficient;
setting parameters x ', y ', z ', and respectively realizing the length, width and channel number of the original input data after feature extraction by a data reconstruction module;
Setting batch_Size and Window_Size, wherein batch_Size refers to the number of each Batch of behavior samples of the multi-channel feature extraction and anomaly monitoring module, and Window_Size refers to the length of sample data;
the input layer of the data reconstruction module in the multi-channel feature extraction and anomaly monitoring module is provided with c 1 Neurons, hidden layer c 2 The output layer of each neuron is c 3 A neuron;
setting the input and output comparison function in the abnormality monitoring module as f, and judging the threshold value as alpha;
setting the input shape of the SE module as x ', y ', z ' which are the output shape parameters of the data reconstruction module respectively, wherein the scaling parameters of the accounting part are SERADIO;
step S71: raw data input
Assuming that the input data is formed by fusion of measured input data of n triaxial acceleration sensors and triaxial angular velocity sensors, the Size is physiological information of batch_Size×Window_Size×6n, window_Size is the data length, and 6n is the number of data channels;
step S72: data input multichannel feature extraction and anomaly monitoring module
The physiological information with the Size of batch_Size×Window_Size×6n is sent to a multi-channel feature extraction and anomaly monitoring module, the physiological information is split into x, y, … and z-channel data, the x, y, … and z-channel data are input to a noise superposition module, the noise superposition module randomly selects channels of the multi-channel data to superpose Gaussian noise, and the channels are input to a data reconstruction module;
The data reconstruction module is a three-layer network model and comprises an input layer, a hidden layer and an output layer, wherein the number of neurons of the input layer is c 1 The number of neurons in the hidden layer is c 2 The output layer has c 3 Neurons, c 1 、c 2 And c 3 The relationship of (a) is formula (II):
c 1 >c 2 ≥c 3 (II)
final output size of
Inputting the characteristic information of the device into an abnormality monitoring module; the anomaly monitoring module multiplexes the network structure of the data reconstruction module, obtains reconstruction errors by comparing the data of the output layer with the data output by the noise superposition module, calculates anomaly scores obtained by the reconstruction errors, compares the anomaly scores with a judgment threshold alpha, and is abnormal when the anomaly scores exceed the judgment threshold alpha; the loss function of the reconstruction error is of formula (III):
L(x,z)＝‖x-z‖ 2 (III)
in the formula (III), x is original input layer data before noise superposition, and z is output layer data;
step S73: data input transposition coding module
Will be of the size of
The feature vector of (a) is input into a behavior recognition module; setting the original input shape as xxyxz, respectively representing the width, height and channel number, and transposing the original input along different axes to obtain data of yxz x and zxx x y; thereby reassigning different channel weights, reading information of three different dimensions,then inputting the transposed and transposed three channel information into an SE module;
Step S74: data input SE module
The SE module is an extrusion and excitation network module, and firstly calculates the channel attention in an effective mode by utilizing global average set characteristics, wherein the channel attention comprises a pooling layer, a full-connection layer, a convolution layer, a full-connection layer and a convolution layer which are sequentially linked;
wherein the attention mechanism combining the average pooling operation and the maximum pooling operation is described as formula (IV):
Wc＝σ(f{w 1,2 }(AP(χ))+f{w 1 ,w 2 }(MP(χ))) (IV)
in the formula (IV), χ is tensor data input to three SE modules,
representing average pooling and maximum pooling operations, respectively; sigma is a sigmod function, w 1 ,w 2 Is a network parameter used for adjusting a scale factor; formula (IV) is described as formula (V):
Wc＝σ(w 2 ReLU(w 1 AP(χ))+w 2 ReLU(w 1 MP(χ)))(V)
step S75: addition polymerization
The three attention channels output by the SE module are aggregated by addition, then Z_Pooling is used to preserve the feature representation, while depth is compressed, and the connected and aggregated feature information is represented as formula (VI):
Z_Pooling(χ)＝[MaxPool 0d (χ),AvgPool 0d (χ)](VI)
in formula (VI), 0d is the data of the 0 th dimension where maximum pooling and average pooling occur;
step S76: layer normalization
Carrying out batch layer normalization on the aggregated output to obtain 1 XyXz output;
step S77: outputting the unit identification result
The physiological information is converted into scalar quantity from vector through the unfolding layer, and the Size of the information Output after passing through the full-connection layer is batch_size multiplied by output_length; classifying and identifying physiological information through a Softmax classifier; inputting physiological information characteristics with information size into a Softmax classifier, solving the probability of each behavior, and obtaining the behavior corresponding to the maximum probability, namely the final behavior recognition result of the multi-channel attention model network based on transposed encoding.
10. The method for wearable remote real-time monitoring of electric power operators based on multimode data according to claim 8 or 9, characterized by comprising the following steps of:
step S51: physiological information denoising
Denoising the physiological information by a wavelet threshold method;
step S52: physiological information normalization
Carrying out mean normalization processing by a mean normalization method to enable the characteristics of different dimensions to be in the same numerical magnitude;
step S53: multimodal data fusion
Aligning the acquired multiple physiological information according to the time stamp;
step S54: time series based interception
And referring to parameters preset by a user, including the size of a sliding window, the cutting frequency and the step length of the sliding window, performing sliding window processing on the physiological information processed in the step S53, so that the physiological information is input into a multichannel attention model network based on transposed encoding in the form of information blocks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310197634.XA CN116382472A (en) | 2023-03-03 | 2023-03-03 | Multi-mode data-based wearable remote real-time monitoring system and method for electric power operators |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310197634.XA CN116382472A (en) | 2023-03-03 | 2023-03-03 | Multi-mode data-based wearable remote real-time monitoring system and method for electric power operators |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116382472A true CN116382472A (en) | 2023-07-04 |
Family
ID=86979663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310197634.XA Pending CN116382472A (en) | 2023-03-03 | 2023-03-03 | Multi-mode data-based wearable remote real-time monitoring system and method for electric power operators |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116382472A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235650A (en) * | 2023-11-13 | 2023-12-15 | 国网浙江省电力有限公司温州供电公司 | Method, device, equipment and medium for detecting high-altitude operation state |
| CN117454166A (en) * | 2023-10-11 | 2024-01-26 | 国网四川省电力公司电力科学研究院 | Method for identifying arc faults of ignition based on EffNet lightweight model |
| CN117454166B (en) * | 2023-10-11 | 2024-05-10 | 国网四川省电力公司电力科学研究院 | Method for identifying arc faults of induced thermal power based on EffNet lightweight model |
-
2023
- 2023-03-03 CN CN202310197634.XA patent/CN116382472A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117454166A (en) * | 2023-10-11 | 2024-01-26 | 国网四川省电力公司电力科学研究院 | Method for identifying arc faults of ignition based on EffNet lightweight model |
| CN117454166B (en) * | 2023-10-11 | 2024-05-10 | 国网四川省电力公司电力科学研究院 | Method for identifying arc faults of induced thermal power based on EffNet lightweight model |
| CN117235650A (en) * | 2023-11-13 | 2023-12-15 | 国网浙江省电力有限公司温州供电公司 | Method, device, equipment and medium for detecting high-altitude operation state |
| CN117235650B (en) * | 2023-11-13 | 2024-02-13 | 国网浙江省电力有限公司温州供电公司 | Method, device, equipment and medium for detecting high-altitude operation state |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110197128A (en) | The recognition of face architecture design method planned as a whole based on edge calculations and cloud | |
| CN110796284A (en) | Method and device for predicting pollution level of fine particulate matters and computer equipment | |
| CN110308104B (en) | Atmospheric pollutant NO2Concentration monitoring method and system | |
| CN116382472A (en) | Multi-mode data-based wearable remote real-time monitoring system and method for electric power operators | |
| CN106936517A (en) | A kind of automatic recognition system and its method of abnormal radio signal | |
| De Medeiros et al. | An IoT-based air quality monitoring platform | |
| CN116539829A (en) | Water quality monitoring sensing analysis method | |
| CN114511784A (en) | Environment monitoring and early warning method, device, equipment and storage medium | |
| CN117095506B (en) | Fire safety monitoring system and method based on alarm area model | |
| CN114423034B (en) | Indoor personnel action recognition method, system, medium, equipment and terminal | |
| CN116738552B (en) | Environment detection equipment management method and system based on Internet of things | |
| CN113115229A (en) | Personnel trajectory tracking method and system based on Beidou grid code | |
| CN114518143A (en) | Intelligent environment sensing system | |
| CN115620471A (en) | Image identification security system based on big data screening | |
| Haji et al. | IoT for smart environment monitoring based on Python: a review | |
| CN114659993A (en) | Greenhouse gas emission detection system based on satellite remote sensing | |
| CN111242028A (en) | Remote sensing image ground object segmentation method based on U-Net | |
| CN114613101A (en) | Intelligent hotel security system based on big data | |
| CN111104639B (en) | Point-surface fused time sequence PM2.5 spatial distribution estimation method | |
| CN110596017B (en) | Hyperspectral image soil heavy metal concentration assessment method based on space weight constraint and variational self-coding feature extraction | |
| CN117156386A (en) | Intelligent positioning, calling for help and alarming intercom equipment for fire control | |
| Sallis et al. | Frost prediction characteristics and classification using computational neural networks | |
| Murugan et al. | AI based Weather Monitoring System | |
| CN110361340B (en) | Atmospheric pollution source SO2 monitoring method and system based on unmanned aerial vehicle | |
| Chang et al. | Safety risk assessment of electric power operation site based on variable precision rough set |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |