CN114041780B - Method for monitoring respiration based on data acquired by inertial sensor - Google Patents
Method for monitoring respiration based on data acquired by inertial sensor Download PDFInfo
- Publication number
- CN114041780B CN114041780B CN202111489019.3A CN202111489019A CN114041780B CN 114041780 B CN114041780 B CN 114041780B CN 202111489019 A CN202111489019 A CN 202111489019A CN 114041780 B CN114041780 B CN 114041780B
- Authority
- CN
- China
- Prior art keywords
- convolution
- data
- respiration
- convolution network
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000029058 respiratory gaseous exchange Effects 0.000 title claims abstract description 127
- 238000012544 monitoring process Methods 0.000 title claims abstract description 84
- 238000000034 method Methods 0.000 title claims description 37
- 238000012545 processing Methods 0.000 claims abstract description 92
- 238000001914 filtration Methods 0.000 claims abstract description 79
- 230000007246 mechanism Effects 0.000 claims abstract description 30
- 230000002708 enhancing effect Effects 0.000 claims abstract description 4
- 230000000241 respiratory effect Effects 0.000 claims description 56
- 238000005070 sampling Methods 0.000 claims description 51
- 230000001133 acceleration Effects 0.000 claims description 25
- 238000012549 training Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 3
- 210000003128 head Anatomy 0.000 description 21
- 238000004422 calculation algorithm Methods 0.000 description 11
- 230000033001 locomotion Effects 0.000 description 11
- 239000011159 matrix material Substances 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000009499 grossing Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- 230000003044 adaptive effect Effects 0.000 description 4
- 230000000295 complement effect Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 235000013290 Sagittaria latifolia Nutrition 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 238000009610 ballistocardiography Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 235000015246 common arrowhead Nutrition 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000007789 gas Substances 0.000 description 2
- 230000035790 physiological processes and functions Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000036391 respiratory frequency Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 210000000214 mouth Anatomy 0.000 description 1
- 210000003928 nasal cavity Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009323 psychological health Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
- A61B2562/02—Details of sensors specially adapted for in-vivo measurements
- A61B2562/0219—Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Pathology (AREA)
- Evolutionary Computation (AREA)
- Animal Behavior & Ethology (AREA)
- Surgery (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Veterinary Medicine (AREA)
- Heart & Thoracic Surgery (AREA)
- Physiology (AREA)
- Data Mining & Analysis (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Pulmonology (AREA)
- Fuzzy Systems (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention provides a respiration monitoring model, which comprises a processing main road and a plurality of processing branches, wherein each processing branch and the main road comprise convolution filters, each convolution filter comprises a plurality of convolution networks for filtering input data and a multi-head self-attention mechanism layer arranged between the corresponding convolution networks for enhancing the global receptive field, and the respiration monitoring model is configured to: respectively inputting each modal data in the multi-modal data obtained based on the inertial sensor data into a corresponding processing branch circuit for convolution filtering to obtain a filtering result of each modal data; and inputting multi-modal respiration characteristics obtained by superposing filtering results of the modal data into a processing main path for convolution filtering to obtain correlation respiration characteristics, and generating a respiration waveform based on the correlation respiration characteristics. The invention generates the respiration waveform through the model, thereby monitoring the human respiration condition.
Description
Technical Field
The invention relates to the field of digital signal processing and deep learning, in particular to a method for monitoring respiration based on data acquired by an inertial sensor.
Background
Respiratory monitoring has a self-evident importance in physiological and psychological health, as well as a significant clinical medical and social significance. However, the respiratory monitoring device, which is a medical instrument in the conventional sense, is expensive and professional, and the invasive airflow detection method is not suitable for long-term wearing. With the development of sensor technology, some civilian respiration monitoring products have been developed. Such as a thoracoabdominal wrap-around respiration monitoring product, which uses pressure sensitive sensors or magnetic flux sensors to monitor respiration-induced thoracoabdominal cavity motion to generate a respiration waveform. There are also respiratory mask type respiratory monitoring products that monitor respiration by wearing a mask over the oral and nasal cavities and sensing the exchange of gases produced by the respiration using a gas flow meter. These methods, although reducing the price of a part of the device compared to medical devices, are still costly, lose part of the precision, and are poorly wearable. Therefore, people cannot wear the wearable device continuously in daily life, and the wearable device cannot be used as a portable wearable device for using the health service provided by the wearable device.
Therefore, respiratory monitoring to achieve universality and versatility has been the goal of the field, and Ballistocardiography (Ballistocardiography) was proposed as early as Starr et al in the 30's of the 20 th century. Similarly, the respiratory movement of the human body is reflected on the limbs of the human body, and weak respiratory movement signals are generated on the limbs. The motion signal may be acquired by using a sensor such as an accelerometer or a gyroscope in an Inertial Measurement Unit (IMU), and filtered to obtain a respiration waveform.
However, the limbs of the human body are accompanied by daily behavioral activity, so that a large amount of motion artifacts are generated in the IMU data, and a weak respiratory signal is largely aliased by the motion artifacts. The IMU data with complex interference conditions generated by the movement can not be comprehensively analyzed and processed by the traditional filtering algorithm, namely filtering is not performed from multiple angles of the multi-modal data, the generalization and the accuracy are poor, and better filtering capability is not provided by utilizing a plurality of convolutional networks. Finally, an existing adaptive filtering algorithm is dependent on partial known performance of an expected respiratory signal to perform parameter adjustment, but since the amplitude, curvature and frequency of respiration are important indexes reflecting physiological states, the amplitude, curvature and frequency of the respiration cannot be predicted in advance, weak change is continuously performed, and a multi-head attention mechanism is not utilized to make up for the problem that local features are too much concerned in the filtering process, so that the respiratory waveform extracted by the adaptive filtering algorithm is low in precision and does not have a global receptive field.
Therefore, a respiration monitoring system based on filtering of data acquired by the IMU is needed to overcome the problems of difficult extraction of weak features and complex interference caused by human body motion, so as to obtain a high-precision respiration waveform.
Disclosure of Invention
It is therefore an object of the present invention to overcome the above-mentioned drawbacks of the prior art and to provide a method for respiratory monitoring based on data acquired by an inertial sensor.
The purpose of the invention is realized by the following technical scheme:
according to a first aspect of the present invention, a respiration monitoring model comprises a processing main and a plurality of processing branches, each of the processing branches and the processing main comprises a convolution filter, each convolution filter comprises a plurality of convolution networks for filtering input data and a multi-head self-attention mechanism layer arranged between the corresponding convolution networks for enhancing a global receptive field, wherein the respiration monitoring model is configured to: respectively inputting each modal data in the multi-modal data obtained based on the data of the inertial sensor into a corresponding processing branch circuit for convolution filtering to obtain a filtering result of each modal data; and inputting multi-modal respiration characteristics obtained by superposing filtering results of the modal data into a processing main path for convolution filtering to obtain correlation respiration characteristics, and generating a respiration waveform based on the correlation respiration characteristics.
In some embodiments of the invention, the convolution filter further comprises a down-sampling layer and an up-sampling layer disposed between respective convolution networks; the down-sampling layer is used for down-sampling a result obtained by filtering input data by the corresponding convolution network to obtain a low-dimensional feature; the up-sampling layer is used for up-sampling a result obtained by filtering the corresponding input low-dimensional features by the corresponding convolution network to obtain high-dimensional features; wherein the low-dimensional features and the high-dimensional features respectively enhance the respective global receptive fields by utilizing a multi-head self-attention mechanism layer.
In some embodiments of the present invention, the convolution filter includes a first convolution network, a second convolution network, a third convolution network, and a fourth convolution network, all of which are one-dimensional convolution networks, wherein an input end of the first convolution network is an input end of the convolution filter, and an output end of the fourth convolution network is an output end of the convolution filter; a multi-head self-attention mechanism layer is arranged between the first convolution network and the fourth convolution network, a down-sampling layer is arranged between the first convolution network and the second convolution network, and a down-sampling layer is arranged between the first convolution network and the third convolution network; an upper sampling layer is arranged between the second convolution network and the fourth convolution network, and a multi-head self-attention mechanism layer is arranged between the second convolution network and the third convolution network; an up-sampling layer is arranged between the third convolution network and the fourth convolution network.
In some embodiments of the invention, the convolution filter is configured to: processing input data sequentially through a first convolution network and a multi-head self-attention mechanism layer to obtain a first high-dimensional feature; processing input data sequentially through a first convolution network, a down-sampling layer, a second convolution network and an up-sampling layer to obtain a second high-dimensional feature; processing input data through a first convolution network and a down-sampling layer in sequence to obtain a first low-dimensional feature; processing input data sequentially through a first convolution network, a down-sampling layer, a second convolution network and a multi-head self-attention mechanism layer to obtain a second low-dimensional feature; overlapping the first low-dimensional feature and the second low-dimensional feature, and sequentially processing the first low-dimensional feature and the second low-dimensional feature through a third convolution network and an upper sampling layer to obtain a third high-dimensional feature; and overlapping the first high-dimensional feature, the second high-dimensional feature and the third high-dimensional feature, and processing through a fourth convolution network to obtain a filtering result.
In some embodiments of the invention, the processing branch or processing head includes one or more convolution filters; when the processing branch or the processing main comprises a plurality of convolution filters, any two convolution filters in the plurality of convolution filters are connected in a longitudinal stacking connection or transverse stacking connection mode, wherein the first convolution filter and the second convolution filter which are connected in the longitudinal stacking connection are connected in a mode that the output end of a second convolution network of the first convolution filter is connected with the input end of the second convolution filter, and the output end of the second convolution filter is connected with the input end of a third convolution network of the first convolution filter; the first convolution filter and the second convolution filter which are connected in a transverse stacking mode are connected in a mode that the output end of the first convolution filter is connected with the input end of the second convolution filter.
According to a second aspect of the invention, there is provided a training method for the respiration monitoring model of the first aspect of the invention, comprising performing a plurality of iterative training of the respiration monitoring model in the following manner: acquiring a training set, wherein input data of a sample in the training set is each modal data in multi-modal data obtained based on data of an inertial sensor corresponding to a corresponding time window, and a label of the sample is a standard respiratory waveform corresponding to the corresponding time window; training a respiration monitoring model by using a training set, and performing convolution filtering on input data to generate a respiration waveform; calculating a total loss value based on a difference of the generated respiration waveform and a standard respiration waveform; and updating parameters of the respiration monitoring model based on the total loss value to obtain a trained respiration monitoring model.
In some embodiments of the present invention, the total loss value is calculated from a huber loss calculated based on the generated respiratory waveform and the standard respiratory waveform and an L2 regularization term by:
wherein n is the total amount of samples, z i Huber loss for the ith sample, equal to
x i Is a respiratory waveform, y, generated based on the ith sample i Is the label of the ith sample, | | x i || 2 Denotes x i 2 norm, | | y i || 2 Denotes y i 2 norm of, max (| x) i || 2 ·||y i || 2 And e) represents taking x i || 2 ·||y i || 2 And the maximum value in e is used as a denominator, e is a scalar value, a is a parameter of the L2 regularization term, and Ω is the L2 regularization term.
According to a third aspect of the present invention, there is provided a system for respiratory monitoring based on data acquired by an inertial sensor, comprising: the data processing module is used for obtaining acceleration modal data, angular velocity modal data and Euler angle modal data based on data collected by an inertial sensor in the wearable device; and processing the acceleration modal data, the angular velocity modal data and the Euler angle modal data by using a respiration monitoring model trained by the method of the second aspect of the invention to generate a respiration waveform.
According to a fourth aspect of the present invention, there is provided a method of respiratory monitoring based on data acquired by an inertial sensor, comprising: obtaining acceleration modal data, angular velocity modal data and Euler angle modal data based on data acquired by an inertial sensor in the wearable device; and processing the acceleration modal data, the angular velocity modal data and the Euler angle modal data by using the respiration monitoring model trained by the method of the second aspect of the invention to generate a respiration waveform.
According to a fifth aspect of the present invention, there is provided an electronic apparatus comprising: one or more processors; and a memory, wherein the memory is to store executable instructions; the one or more processors are configured to implement the steps of the method of any of the second and fourth aspects of the invention via execution of the executable instructions.
Compared with the prior art, the invention has the advantages that:
1. according to the respiration monitoring model, firstly, convolution filters of each processing branch circuit are used for performing convolution filtering on each modal data, wherein each modal data is obtained by processing inertial sensor data, the characteristics of multi-modal data of weak inertial sensor data are enhanced, and meanwhile convolution filtering is performed from each angle of the enhanced multi-modal data, so that overfitting and lack of generalization are avoided; secondly, the convolution filter of the processing main road performs convolution filtering on the multi-modal respiration characteristics obtained by superposing the filtering results of the modal data, the filtering results of the multi-modal data are fused to generate a respiration waveform, the model accuracy is improved, finally, the convolution filters of the processing branch road and the processing main road perform convolution filtering on the basis of a convolution network, the filtering capability is enhanced, and the overall receptive field of the respiration waveform is enhanced by using a multi-head self-attention mechanism.
2. The convolution filter in the model realizes multiple coding and decoding of the input data by using a very small number of convolution networks and combining a multi-head self-attention mechanism layer, a down-sampling layer and an up-sampling layer when carrying out convolution filtering on the input data, avoids overfitting and lack of generalization of the model again and realizes better filtering capability. In addition, the convolution filter has better modularization property and expandable performance, and a plurality of convolution filters can be connected in a vertical stacking mode and in a horizontal stacking mode according to task requirements. The longitudinal stacking connection can increase the number of down-sampling layers and the number of up-sampling layers, can realize deeper coding and decoding, and enhances the filtering capability. The lateral stacking connections may increase the number of fits to the feature, enabling a better fit to the feature.
3. The respiration monitoring method combines low-pass filtering and an attitude calculation complementary filtering algorithm to fuse acceleration modal data and angular velocity modal data, enhances the characteristics of multi-modal data of inertial sensor data, enables a respiration monitoring model to better extract the characteristics, and utilizes a mean value smoothing module to continuously deduce and optimize respiration waveforms covering amplitude, curvature and frequency changes, so that the waveforms are smoother.
Drawings
Embodiments of the invention are further described below with reference to the accompanying drawings, in which:
FIG. 1 is a schematic diagram of a system for performing respiratory monitoring in accordance with one embodiment of the present invention;
FIG. 2 is a schematic diagram of a respiratory monitoring model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a convolution filter according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a structure of a one-dimensional convolution network in a convolution filter according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a multi-headed self-attentive machining layer according to one embodiment of the invention;
FIG. 6 is a schematic diagram of a respiratory monitoring model according to another embodiment of the present invention;
FIG. 7 is a schematic diagram of a plurality of convolution filters connected in a vertical stack according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a plurality of convolution filters connected in a lateral stack according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a respiration waveform and a true respiration waveform output by a respiration monitoring model according to one embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail by embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As mentioned in the background section, the limbs of the person are accompanied by activities of daily behavior, so that a large amount of motion artifacts are generated in the IMU data, and weak respiratory signals are largely aliased by the motion artifacts. The IMU data with complex interference conditions generated by the movement cannot be comprehensively analyzed and processed by the traditional filtering algorithm, namely filtering is not performed from multiple angles of multi-modal data, the generalization and the accuracy are low, and better filtering capability is not provided by utilizing a plurality of convolutional networks. Finally, an existing adaptive filtering algorithm is dependent on partial known performance of an expected respiratory signal to perform parameter adjustment, but since the amplitude, curvature and frequency of respiration are important indexes reflecting physiological states, the amplitude, curvature and frequency of the respiration cannot be predicted in advance, weak change is continuously performed, and a multi-head attention mechanism is not utilized to make up for the problem that local features are too much concerned in the filtering process, so that the respiratory waveform extracted by the adaptive filtering algorithm is low in precision and does not have a global receptive field.
Aiming at the defects of the existing method, according to one embodiment of the invention, a system for monitoring respiration based on data acquired by an inertial sensor is provided, which comprises a data processing module, a respiration monitoring model and a mean value smoothing module, and firstly, accelerometer data and gyroscope data of the inertial sensor worn on a human body are acquired through the data processing module, the two data are processed, the characteristics of each data of the inertial sensor are enhanced in the processing process, and acceleration modal data, angular velocity modal data and Euler angle modal data are acquired; secondly, carrying out convolution filtering processing on each modal data through convolution filters of corresponding processing branches 2 of the respiration monitoring model so as to realize multi-angle filtering of the multi-modal data and avoid overfitting and lack of generalization of the respiration monitoring model, carrying out convolution filtering processing on multi-modal respiration characteristics obtained by superposing filtering results of the various modal data through the convolution filters arranged in the processing main 1 so as to obtain relevance respiration characteristics, generating a respiration waveform based on the relevance respiration characteristics, and improving model accuracy; and finally, obtaining the optimized respiration waveform through a mean value smoothing module.
Furthermore, convolution filters arranged on each processing branch 2 and each processing main 1 carry out convolution filtering on the basis of a plurality of convolution networks, the filtering capability is enhanced, and the overall receptive field of the respiratory waveform is enhanced by using a multi-head self-attention mechanism layer.
Before describing embodiments of the present invention in detail, some of the terms used therein will be explained as follows:
high dimensional characteristics: the method includes a plurality of features irrelevant to a breathing monitoring model learning task (for example, a plurality of features with only weak correlation exist), a plurality of features redundant to the breathing monitoring model learning task (for example, features with strong correlation exist), noise data and other features. For example, the original features of the modal data and the result of convolution filtering the modal data by the convolution filter are high-dimensional features.
Low dimensional features: the method is characterized in that high-dimensional features are subjected to dimension reduction (features irrelevant to a breathing monitoring model learning task and partially redundant features are removed) to obtain the features. According to the invention, the original features of the modal data are down-sampled so as to reduce the dimension of the original features, and the obtained features are low-dimensional features.
For a better understanding of the present invention, the following detailed description is directed to the structure of each part of the respiration monitoring model in conjunction with specific embodiments.
According to an embodiment of the present invention, a respiration monitoring model is provided, see fig. 2, which comprises a processing main 1 and three processing branches 2, wherein each processing branch 2 and the processing main 1 are provided with convolution filters. The output ends of the three processing branches 2 superpose the output results in a cascading manner and input the superposed output results to the input end of the processing main 1. And in the form of the input portion of the three processing branches 2, each data in the array of (N, 3, 256) on the arrow indicates that the corresponding data is input with the corresponding batch number N, the channel number 3 and the length 256. An array with an arrow head (N, 8, 256) in the processing branch 2 indicates that the corresponding batch number is N, the number of channels is 8, and the length is 256 when the corresponding data is output, and an array with an arrow head (N, 8, 128) indicates that the corresponding batch number is N, the number of channels is 8, and the length is 128 when the corresponding data is input.
According to one embodiment of the invention, the respiration monitoring model is configured to: respectively inputting each modal data in the multi-modal data obtained based on the inertial sensor data into the corresponding processing branch 2 for convolution filtering to obtain a filtering result of each modal data; and inputting multi-modal respiration characteristics obtained by superposing filtering results of the modal data into the processing main 1 for convolution filtering to obtain correlation respiration characteristics, and generating a respiration waveform based on the correlation respiration characteristics.
According to an embodiment of the present invention, each convolution filter in the processing trunk 1 and the plurality of processing branches 2 includes a plurality of convolution networks for filtering input data, a multi-head attention mechanism layer disposed between the respective convolution networks for enhancing a global receptive field, and a down-sampling layer and an up-sampling layer disposed between the respective convolution networks. Referring to fig. 3, the specific structure of each convolution filter may be: the convolution networks comprise a first convolution network, a second convolution network, a third convolution network and a fourth convolution network which are all one-dimensional convolution networks, wherein the input end of the first convolution network is the input end of the convolution filter, and the output end of the fourth convolution network is the output end of the convolution filter. A multi-head self-attention mechanism layer is arranged between the first convolution network and the fourth convolution network, a downsampling layer is arranged between the first convolution network and the second convolution network, and a downsampling layer is arranged between the third convolution network and the third convolution network; an upper sampling layer is arranged between the second convolution network and the fourth convolution network, and a multi-head self-attention mechanism layer is arranged between the second convolution network and the third convolution network; an upper sampling layer is arranged between the third convolution network and the fourth convolution network.
According to one embodiment of the invention, the downsampling layer of the convolution filter is used for downsampling a result obtained by filtering input data by a corresponding convolution network to obtain a low-dimensional feature; and the up-sampling layer of the convolution filter is used for up-sampling a result obtained by filtering the corresponding input low-dimensional features by the corresponding convolution network to obtain high-dimensional features, wherein the low-dimensional features and the high-dimensional features respectively enhance the respective global receptive fields by utilizing the multi-head self-attention mechanism layer.
According to one embodiment of the invention, the down-sampling layer adopts a maximum pooling strategy, the up-sampling layer adopts a linear interpolation algorithm, wherein the down-sampling layer is arranged behind the output end of the corresponding convolution network, the maximum pooling strategy is used to obtain the low-dimensional characteristics retaining the core characteristics, the method is beneficial to the characteristic abstraction work of complex signals, the maximum value of the region is selected for the output result of the corresponding convolution network, the time invariance characteristic discovery of the convolution kernel is effectively helped, and the robustness is increased. The linear interpolation algorithm adopted by the down-sampling layer expands the low-dimensional features to obtain the high-dimensional features, has the property of low-pass filtering, and focuses more on the core features of the low-dimensional features, so that the finally generated respiratory waveform is smoother.
According to an embodiment of the present invention, each convolution network includes a multi-layer structure, for example, referring to fig. 4, each convolution network includes a one-dimensional convolution layer, a normalized operation layer, a Linear rectification activation function layer (ReLU), a one-dimensional convolution layer, a normalized operation layer, and a Linear rectification activation function layer, which are connected in sequence.
According to one embodiment of the present invention, referring to FIG. 5, the multi-headed self-Attention mechanism layer includes h self-Attention heads (Attention) i ) And a full linkAnd the layer is connected, the output ends of all the self-attention heads are cascaded, and the multi-head self-attention mechanism is formed by connecting the output ends of all the self-attention heads with the input end of the full connection layer, and the principle is as follows:
and each self-attention head carries out linear transformation on the input features F, and carries out feature attention weighting on the h self-attention heads to obtain a processing result head of each self-attention head.
If the ith self-attention layer carries out linear transformation on the input characteristic F, obtaining
Indexing:
is a function of the corresponding weight matrix and,
is a corresponding bias matrix;
bond:
is a function of the corresponding weight matrix and,
is a corresponding bias matrix;
the value:
is a function of the corresponding weight matrix and,
is a corresponding bias matrix;
carrying out attention weighting on the features of the h self-attention heads to obtain a processing result head of each self-attention head i ＝Attention(Q i ，K i ，V i ) The following are:
wherein,
for inputting key K i The dimension (c) of (a) is,
is a V i Attention weight matrix, coefficients
For normalizing the attention weight matrix and finally, V by softmax layer i Attention weighting is performed.
The outputs of the h self-attention head heads are cascaded and input to the full-connection layer for processing, and a final output result MultiHead (Q, K, V) of the multi-head self-attention mechanism layer is obtained as follows:
MultiHead(Q，K，V)＝Concat(head 1 ，…，head h )W O +B O ；
q represents a final index obtained by combining h indexes obtained by respectively performing linear transformation on the features by h self-attention heads, K represents a final key obtained by combining h keys obtained by respectively performing linear transformation on the features by the h self-attention heads, V represents a final value obtained by combining h values obtained by respectively performing linear transformation on the features by the h self-attention heads, and Concat represents that head is obtained by combining h values 1 ，...，head h Are combined into a character string, W O Is the corresponding weight matrix, B, linearly transformed by each of the self-attentional heads O Is the corresponding bias matrix that each self-attention head performs a linear transformation.
The multi-head self-attention mechanism layer of the convolution filter has global analysis capability, for example, aiming at the characteristics of long breathing period and weak characteristics, the global time sequence correlation analysis of a long sequence is carried out, so that the defect that the structure with a plurality of convolution networks is too much concerned about local characteristics and lacks global receptive field is overcome. And the structure with a plurality of convolutional networks can also make up the defect that a multi-head self-attention mechanism layer is suitable for high-semantic sequence feature processing and provides better filtering capability for poor low-semantic multi-mode data processing effect. The multi-mode data with low semantics are filtered by utilizing a plurality of convolutional networks to obtain sequence features with high semantics, and a multi-head self-attention mechanism layer is utilized to discover global time sequence relevance feature signals to enhance the global receptive field.
According to an embodiment of the invention, the convolution filter further comprises a function of realizing multiple coding and decoding of the input data by using a very small number of convolution networks and combining a down-sampling layer and an up-sampling layer when filtering the input data, so that model overfitting and lack of generalization are avoided again, and the filtering capability is enhanced. Preferably, the convolution filter filters the input data in the following manner:
the convolution filter sequentially processes input data through a first convolution network and a multi-head self-attention mechanism layer to obtain a first high-dimensional feature.
The first codec process for performing multiple codecs is: the convolution filter processes input data through the first convolution network, the down-sampling layer, the second convolution network and the up-sampling layer in sequence to obtain a second high-dimensional characteristic.
The second encoding and decoding process is carried out as follows: the convolution filter processes input data through a first convolution network and a down-sampling layer in sequence to obtain a first low-dimensional feature; processing input data sequentially through a first convolution network, a down-sampling layer, a second convolution network and a multi-head self-attention mechanism layer to obtain a second low-dimensional feature; and overlapping the first low-dimensional feature and the second low-dimensional feature, and sequentially processing the first low-dimensional feature and the second low-dimensional feature through a third convolution network and an upper sampling layer to obtain a third high-dimensional feature.
And the convolution filter superposes the first high-dimensional feature, the second high-dimensional feature and the third high-dimensional feature and then processes the superposed features through a fourth convolution network to obtain a filtering result.
According to one embodiment of the invention, referring to fig. 6, each processing branch 2 of the respiration monitoring model is provided with one convolution filter, the processing main 1 is provided with two convolution filters, and the two convolution filters are connected in a longitudinally stacked connection. Referring to fig. 7, the output end of the second convolution network including the first convolution filter is connected to the input end of the second convolution filter, and the output end of the second convolution filter is connected to the input end of the third convolution network of the first convolution filter. By the longitudinal stacking connection mode, the number of down-sampling layers and the number of up-sampling layers are increased, and deeper coding and decoding can be realized.
According to another embodiment of the invention, the two convolution filters of the processing manifold 1 may also be connected in a horizontally stacked connection, see fig. 8, i.e. the first convolution filter and the second convolution filter are connected in such a way that the output of the first convolution filter is interconnected with the input of the second convolution filter. The lateral stacking connections may increase the number of fits to the feature, enabling a better fit to the feature.
According to another embodiment of the invention, the convolution filter has better modularization property and expandable performance, and can be used for vertical stacking connection and horizontal stacking connection according to task requirements. The processing branch 2 of the respiration monitoring model can be set as a convolution filter, or a plurality of convolution filters connected in a vertical stack and/or a horizontal stack, and the processing main 1 can also be set as a convolution filter, or a plurality of convolution filters connected in a vertical stack and/or a horizontal stack.
According to another embodiment of the present invention, when a plurality of convolution filters are disposed in the processing branch 2 or the processing main 1, wherein any two convolution filters of the plurality of convolution filters are connected in a vertically stacked connection or a horizontally stacked connection, if the processing main 1 is disposed with three convolution filters, the first convolution filter is connected to the second convolution filter in a vertically stacked connection, and the second convolution filter is connected to the third convolution filter in a horizontally stacked connection; or the three convolution filters are all transversely stacked and connected; or both may be longitudinally stacked.
According to an embodiment of the invention, the three processing branches 2 of the respiration monitoring model respectively perform convolution filtering on acceleration modal data, angular velocity modal data and euler angle modal data obtained by processing data of the inertial sensor. The multi-angle discovery of the breathing characteristics of the multi-mode data is carried out, so that the phenomenon that the breathing monitoring model is difficult to extract too weak characteristics, overfitting and lack of generalization are avoided, and meanwhile, the filtering capability of the breathing monitoring model is enhanced.
According to one embodiment of the invention, the respiratory monitoring model is trained with a large amount of sample data to ensure its accuracy. According to one embodiment of the invention, the sample may be obtained by:
each data frame is marked with a timestamp when the data of each modality is acquired and the standard respiratory waveform is acquired, and the standard respiratory waveform and the data of each modality are segmented into data segments with a certain time window size (for example 256 data frames) after the time and data frame alignment is carried out, so that a training set is obtained. And the standard respiration waveform corresponding to the corresponding time window is used as a label of the sample, and the modal data corresponding to the corresponding time window is used as the input data of the sample. Wherein each sample comprises 256 sets of input data as follows:
(a x ，a y ，a z ，g x ，g y ，g z ，a′ x ，a′ y ，a′ z ) And each group of data corresponds to acceleration modal data of three channels of an X axis, a Y axis and a Z axis at corresponding time: a is x ，a y ，a z (ii) a Angular velocity modal data of three channels of X-axis, Y-axis and Z-axis at corresponding time: g x ，g y ，g z (ii) a Euler angle modal data at the respective time: a' x ，a′ y ，a′ z 。
According to an embodiment of the present invention, after obtaining a training set, a respiration monitoring model is trained through the training set, and parameters of the model are adjusted to enable the finally obtained respiration monitoring model to output a more accurate respiration waveform, and a method for training a respiration monitoring model through the training set obtained above is provided to obtain a better respiration monitoring model, and the method includes performing iterative training on the respiration monitoring model for a plurality of times according to the following steps a1, a2, a3 and a 4:
step a 1: and acquiring a training set, wherein input data of a sample in the training set is each modal data in multi-modal data obtained based on data of the inertial sensor corresponding to a corresponding time window, and a label of the sample is a standard respiratory waveform corresponding to the corresponding time window.
Step a 2: and carrying out convolution filtering on the input data by utilizing a training set to train a respiration monitoring model to generate a respiration waveform.
Step a 3: a total loss value is calculated based on a difference of the generated respiratory waveform and the standard respiratory waveform.
In some embodiments of the invention, the total loss value is obtained from a huber loss calculated based on the generated respiratory waveform and the standard respiratory waveform and an L2 regularization term calculation, the huber loss and the cosine distance are combined, the huber loss contributes to fitting of outliers and prevents gradient explosion, and the cosine distance contributes to fitting of overall waveform characteristics. Meanwhile, to avoid overfitting the model, an L2 regularization term may be used. The calculation method is as follows:
wherein n is the total amount of samples, z i Huber loss for the ith sample, equal to
x i Is a respiratory waveform, y, generated based on the ith sample i Is the label of the ith sample, | | x i || 2 Denotes x i 2 norm, | | y i || 2 Denotes y i 2 norm of, max (| x) i || 2 ·||y i || 2 And e) represents taking x i || 2 ·||y i || 2 And e is a scalar value and equal to 1e-8, to prevent the case where the denominator is 0, α is a parameter of the L2 regularization term, and Ω is the L2 regularization term.
Step a 4: and updating parameters of the respiration monitoring model based on the total loss value to obtain a trained respiration monitoring model. And stopping updating the parameters until the preset iteration times or the total loss value is in a preset range, and obtaining the trained respiration monitoring model.
Through the training of the respiration monitoring model, the finally obtained trained respiration monitoring model can be used for respiration monitoring. According to an embodiment of the present invention, a system for respiratory monitoring based on data acquired by an inertial sensor is provided, which includes a data processing module, a trained respiratory monitoring model and a mean value smoothing module, and the following describes each part of the system in detail with reference to specific embodiments.
According to one embodiment of the invention, the data processing module is used for obtaining acceleration modal data, angular velocity modal data and Euler angle modal data based on data collected by an inertial sensor in the wearable device.
According to one embodiment of the invention, the wearable device is worn on a human body, accelerometer data and gyroscope data of an inertial sensor in the wearable device are obtained, low-pass filtering and attitude calculation are carried out on the accelerometer data to obtain acceleration modal data, low-pass filtering and attitude calculation are carried out on the gyroscope data to obtain angular velocity modal data, and the acceleration modal data and the angular velocity modal data are processed through an attitude calculation complementary filtering algorithm to obtain Euler angle modal data.
According to one embodiment of the invention, the accelerometer data comprises three channels of X-axis, Y-axis and Z-axis accelerometer data, and the gyroscope data comprises three channels of X-axis, Y-axis and Z-axis gyroscope data.
And using a low-pass filter to perform low-pass filtering on the accelerometer data on the three channels by taking 3-5 times of the maximum respiratory frequency as a cut-off frequency to obtain the accelerometer data with low frequency, then performing attitude calculation to obtain the acceleration directions of three channels of an X axis, a Y axis and a Z axis, and using the acceleration directions of the three channels as acceleration modal data.
And using a low-pass filter to perform low-pass filtering on the gyroscope data on the three channels by taking 3-5 times of the maximum respiratory frequency as a cut-off frequency to obtain the gyroscope data with low frequency, performing attitude calculation to obtain the angular velocities of three channels, namely an X axis, a Y axis and a Z axis, and using the angular velocities of the three channels as angular velocity modal data.
Based on the data acceleration direction and angular velocity on each channel, an attitude solution complementary filtering algorithm is used, as in equation (1):
angle i ＝K 1 *(angle i-1 +gyro i *dt)+K 2 *accel i (1)；
wherein, K 1 Is an angle (angle) i-1 +gyro i Dt) of the weight coefficient, K 2 Is corresponding item acel i Weight coefficient of (A), and (K) 1 +K 2 )＝1，angle i Is the Euler angle, accel, at time i i Is the acceleration direction at time i, gyro i Is the angular velocity at the i-th instant.
The formula (1) continuously uses the integral of the angular velocity in the acceleration direction for calibration so as to avoid continuous accumulation of the sensor error of the gyroscope in the integral, and finally obtains the Euler angle characteristics after filtering as Euler angle mode data, wherein the Euler angle mode data comprise a Roll angle (Roll), a Pitch angle (Pitch) and a Yaw angle (Yaw). And respectively inputting the acceleration modal data, the angular velocity modal data and the Euler angle modal data into corresponding processing branches 2 of the respiration monitoring model for convolution filtering.
The respiration monitoring model trained based on the training method is used for processing input Euler angle modal data, acceleration modal data and angular velocity modal data to generate a respiration waveform.
After the respiratory monitoring model performs convolution filtering on the above modal data, the generated respiratory waveform contains noise data, and the respiratory waveform is flow data-oriented and has continuity, so that the respiratory waveform generated by the respiratory monitoring model needs to be denoised to obtain an optimized respiratory waveform. According to an embodiment of the present invention, the mean value smoothing module is configured to, at each time, take a mean value of all candidate points at the current time as a final amplitude of the respiratory waveform at the time, so as to perform denoising processing on the respiratory waveform covering amplitude, curvature, and frequency variation generated by the respiratory monitoring model, and obtain an optimized respiratory waveform.
Based on the above system for respiratory monitoring, according to an embodiment of the present invention, there is provided a respiratory monitoring method, including steps b1, b2, and b 3:
step b 1: and obtaining acceleration modal data, angular velocity modal data and Euler angle modal data based on data acquired by an inertial sensor in the wearable equipment.
Step b 2: the well-trained respiration monitoring model is obtained by the method for training the respiration monitoring model, and the input Euler angle modal data, acceleration modal data and angular velocity modal data are processed to generate a respiration waveform.
Step b 3: and optimizing the respiratory waveform by using the mean value smoothing module to obtain the respiratory waveform with smoother sliding receptive field.
Fig. 9 shows a comparison result between the respiration waveform output by the respiration monitoring model and the real respiration waveform of the professional respiration monitor, the ordinate represents the amplitude of the respiration waveform, the abscissa represents the corresponding time, the respiration waveform output by the respiration monitoring model and the real respiration waveform have the same or similar amplitudes at the same time, and the curvature of the whole respiration waveform is mostly similar.
It should be noted that, although the steps are described in a specific order, the steps are not necessarily performed in the specific order, and in fact, some of the steps may be performed concurrently or even in a changed order as long as the required functions are achieved.
The present invention may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.
The computer readable storage medium may be a tangible device that holds and stores the instructions for use by the instruction execution device. The computer readable storage medium may include, for example, but is not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (11)
1. A device based on a respiration monitoring model is characterized in that the respiration monitoring model comprises a processing main path and a plurality of processing branch paths,
each processing branch and the processing main road comprise convolution filters, each convolution filter comprises a plurality of convolution networks for filtering input data and a multi-head self-attention mechanism layer arranged between the corresponding convolution networks for enhancing the global receptive field,
wherein the respiratory monitoring model is configured to:
respectively inputting each modal data in the multi-modal data obtained based on the data of the inertial sensor into a corresponding processing branch circuit for convolution filtering to obtain a filtering result of each modal data; and
and inputting multi-modal breathing characteristics obtained by superposing filtering results of the modal data into a processing main path for convolution filtering to obtain correlation breathing characteristics, and generating a breathing waveform based on the correlation breathing characteristics.
2. The apparatus of claim 1, wherein the convolution filter further comprises a downsampling layer and an upsampling layer disposed between respective convolution networks;
the down-sampling layer is used for down-sampling a result obtained by filtering input data by a corresponding convolution network to obtain a low-dimensional feature;
the up-sampling layer is used for up-sampling a result obtained by filtering the corresponding input low-dimensional features by the corresponding convolution network to obtain high-dimensional features;
wherein the low-dimensional features and the high-dimensional features respectively enhance the respective global receptive fields by utilizing a multi-head self-attention mechanism layer.
3. The apparatus of claim 1 or 2, wherein the convolution filter comprises a first convolution network, a second convolution network, a third convolution network and a fourth convolution network, all of which are one-dimensional convolution networks, wherein an input end of the first convolution network is an input end of the convolution filter, and an output end of the fourth convolution network is an output end of the convolution filter;
a multi-head self-attention mechanism layer is arranged between the first convolution network and the fourth convolution network, a down-sampling layer is arranged between the first convolution network and the second convolution network, and a down-sampling layer is arranged between the first convolution network and the third convolution network;
an upper sampling layer is arranged between the second convolution network and the fourth convolution network, and a multi-head self-attention mechanism layer is arranged between the second convolution network and the third convolution network;
an upper sampling layer is arranged between the third convolution network and the fourth convolution network.
4. The apparatus of claim 3, wherein the convolution filter is configured to:
processing input data sequentially through a first convolution network and a multi-head self-attention mechanism layer to obtain a first high-dimensional feature;
processing input data sequentially through a first convolution network, a down-sampling layer, a second convolution network and an up-sampling layer to obtain a second high-dimensional feature;
processing input data through a first convolution network and a down-sampling layer in sequence to obtain a first low-dimensional feature;
processing input data sequentially through a first convolution network, a down-sampling layer, a second convolution network and a multi-head self-attention mechanism layer to obtain a second low-dimensional feature;
overlapping the first low-dimensional feature and the second low-dimensional feature, and sequentially processing the first low-dimensional feature and the second low-dimensional feature through a third convolution network and an upper sampling layer to obtain a third high-dimensional feature;
and superposing the first high-dimensional feature, the second high-dimensional feature and the third high-dimensional feature, and processing the superposed high-dimensional features through a fourth convolution network to obtain a filtering result.
5. The apparatus of claim 3, wherein the processing branch or processing head comprises one or more convolution filters;
when the processing branch or the processing main comprises a plurality of convolution filters, any two convolution filters in the plurality of convolution filters are connected in a vertical stacking connection mode or a horizontal stacking connection mode,
the first convolution filter and the second convolution filter which are longitudinally stacked and connected are connected in a mode that the output end of a second convolution network of the first convolution filter is connected with the input end of the second convolution filter, and the output end of the second convolution filter is connected with the input end of a third convolution network of the first convolution filter;
the first convolution filter and the second convolution filter which are connected in a transverse stacking mode are connected in a mode that the output end of the first convolution filter is connected with the input end of the second convolution filter.
6. A training method for the respiration monitoring model according to any one of claims 1 to 5, comprising performing a plurality of iterative training of the respiration monitoring model in the following manner:
acquiring a training set, wherein input data of a sample in the training set is each modal data in multi-modal data which is obtained based on data of an inertial sensor and corresponds to a corresponding time window, and a label of the sample is a standard respiration waveform corresponding to the corresponding time window;
training a respiration monitoring model by using a training set, and performing convolution filtering on input data to generate a respiration waveform;
calculating a total loss value based on a difference of the generated respiration waveform and a standard respiration waveform;
and updating parameters of the respiration monitoring model based on the total loss value to obtain a trained respiration monitoring model.
7. The method of claim 6, wherein the total loss value is calculated for a huber loss calculated based on the generated respiratory waveform and a standard respiratory waveform and an L2 regularization term by:
wherein n is the total amount of samples, z i Huber loss for the ith sample, equal to
x i Is a respiratory waveform, y, generated based on the ith sample i Is the label of the ith sample, | | x i || 2 Denotes x i 2 norm, | | y i || 2 Denotes y i 2 norm of, max (| | x) i || 2 ·||y i || 2 And e) represents taking x i || 2 ·||y i || 2 And the maximum value in e is used as a denominator, e is a scalar value, a is a parameter of the L2 regularization term, and Ω is the L2 regularization term.
8. A system for respiratory monitoring based on data collected by inertial sensors, comprising:
the data processing module is used for obtaining acceleration modal data, angular velocity modal data and Euler angle modal data based on data collected by an inertial sensor in the wearable device;
processing the acceleration modal data, the angular velocity modal data, and the euler angular modal data using a respiration monitoring model trained using the method of claim 6 or 7 to generate a respiration waveform.
9. A method for respiratory monitoring based on data collected by an inertial sensor, comprising:
obtaining acceleration modal data, angular velocity modal data and Euler angle modal data based on data acquired by an inertial sensor in the wearable device;
processing the acceleration mode data, the angular velocity mode data and the Euler angle mode data by using a respiration monitoring model trained by the method of claim 6 or 7 to generate a respiration waveform.
10. A computer-readable storage medium, having stored thereon a computer program executable by a processor for carrying out the steps of the method according to any one of claims 6-7 and 9.
11. An electronic device, comprising:
one or more processors; and
a memory, wherein the memory is to store executable instructions;
the one or more processors are configured to implement the steps of the method of any of claims 6-7 and 9 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111489019.3A CN114041780B (en) | 2021-12-08 | 2021-12-08 | Method for monitoring respiration based on data acquired by inertial sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111489019.3A CN114041780B (en) | 2021-12-08 | 2021-12-08 | Method for monitoring respiration based on data acquired by inertial sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114041780A CN114041780A (en) | 2022-02-15 |
| CN114041780B true CN114041780B (en) | 2022-08-30 |
Family
ID=80212419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111489019.3A Active CN114041780B (en) | 2021-12-08 | 2021-12-08 | Method for monitoring respiration based on data acquired by inertial sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114041780B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114652297A (en) * | 2022-03-07 | 2022-06-24 | 骏楷国际有限公司 | Respiration rate detection method and device, mobile terminal and storage medium |
| CN115486833B (en) * | 2022-08-22 | 2023-06-06 | 华南师范大学 | Respiratory state detection method, respiratory state detection device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109805898A (en) * | 2019-03-22 | 2019-05-28 | 中国科学院重庆绿色智能技术研究院 | Critical illness Mortality Prediction method based on attention mechanism timing convolutional network algorithm |
| CN112006654A (en) * | 2020-07-16 | 2020-12-01 | 中国科学技术大学 | Bone age assessment method based on convolutional neural network and multiple attention mechanism |
| CN112806977A (en) * | 2021-02-05 | 2021-05-18 | 复旦大学 | Physiological parameter measuring method based on multi-scale fusion network |
| CN113229825A (en) * | 2021-06-22 | 2021-08-10 | 郑州大学 | Deep neural network-based multi-label multi-lead electrocardiogram classification method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11145293B2 (en) * | 2018-07-20 | 2021-10-12 | Google Llc | Speech recognition with sequence-to-sequence models |
| US11011154B2 (en) * | 2019-02-08 | 2021-05-18 | Tencent America LLC | Enhancing hybrid self-attention structure with relative-position-aware bias for speech synthesis |
-
2021
- 2021-12-08 CN CN202111489019.3A patent/CN114041780B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109805898A (en) * | 2019-03-22 | 2019-05-28 | 中国科学院重庆绿色智能技术研究院 | Critical illness Mortality Prediction method based on attention mechanism timing convolutional network algorithm |
| CN112006654A (en) * | 2020-07-16 | 2020-12-01 | 中国科学技术大学 | Bone age assessment method based on convolutional neural network and multiple attention mechanism |
| CN112806977A (en) * | 2021-02-05 | 2021-05-18 | 复旦大学 | Physiological parameter measuring method based on multi-scale fusion network |
| CN113229825A (en) * | 2021-06-22 | 2021-08-10 | 郑州大学 | Deep neural network-based multi-label multi-lead electrocardiogram classification method |
Non-Patent Citations (2)
| Title |
|---|
| Multimodal Sleep Signals-Based Automated Sleep Arousal Detection;GUANGXIN ZHOU et.al;《IEEE Access》;20200605;第106157-106164页 * |
| 融合卷积神经网络与双向GRU的文本情感分析胶囊模型;程艳 等;《中文信息学报》;20210531;第118-129页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114041780A (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114041780B (en) | Method for monitoring respiration based on data acquired by inertial sensor | |
| Karmore et al. | IoT-based humanoid software for identification and diagnosis of COVID-19 suspects | |
| Zhu et al. | Using deep learning for energy expenditure estimation with wearable sensors | |
| CN105943012B (en) | Heart rate measuring method for removing motion noise in photoplethysmography signals | |
| CN113221726A (en) | Hand posture estimation method and system based on visual and inertial information fusion | |
| Onyekpe et al. | A quaternion gated recurrent unit neural network for sensor fusion | |
| Kim et al. | Nine-Axis IMU-based Extended inertial odometry neural network | |
| CN114459469B (en) | Multi-motion state navigation method and device and intelligent wearable equipment | |
| Duan et al. | A new quaternion-based Kalman filter for human body motion tracking using the second estimator of the optimal quaternion algorithm and the joint angle constraint method with inertial and magnetic sensors | |
| CN116975600B (en) | Biomedical time sequence signal application method for multi-scale and multi-mode contrast learning | |
| Luo et al. | Deep motion network for freehand 3D ultrasound reconstruction | |
| Wang | Multisensor data fusion of motion monitoring system based on BP neural network | |
| Mezari et al. | Gesture recognition using symbolic aggregate approximation and dynamic time warping on motion data | |
| Wang et al. | Arbitrary spatial trajectory reconstruction based on a single inertial sensor | |
| Yuan et al. | A simple self-supervised imu denoising method for inertial aided navigation | |
| Rahman et al. | Upper limb joint angle estimation using wearable IMUs and personalized calibration algorithm | |
| CN117457162A (en) | Emergency call sub-diagnosis method and system based on multi-encoder and multi-mode information fusion | |
| Kraft et al. | Wrist-worn accelerometer based fall detection for embedded systems and IoT devices using deep learning algorithms | |
| Brückner et al. | Modification and fixed-point analysis of a Kalman filter for orientation estimation based on 9D inertial measurement unit data | |
| CN114504298B (en) | Physiological characteristic discriminating method and system based on multisource health perception data fusion | |
| Chang et al. | 6DoF tracking in virtual reality by deep RNN model | |
| CN115859221A (en) | Human body activity recognition method based on multi-position sensor | |
| CN113822420B (en) | Dead space fraction model building method and system based on volumetric carbon dioxide diagram | |
| Cao | Analysis of aerobic training posture using machine vision for body area networks | |
| Zhou et al. | Fully convolutional neural network and PPG signal for arterial blood pressure waveform estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |