DE102020117244A1 - IoT-based intelligent bed sensor system for contactless respiratory rate monitoring - Google Patents

Abstract

The invention provides a method for determining a patient's respiratory rate (RR), the method comprising: measuring a patient's weight by retrieving weight-related information from a plurality of bed-mounted load sensors (2), transmitting the weight-related information to a server.

Description

area

The present invention relates to a bed-mounted sensor system for contactless respiratory rate monitoring and a corresponding method.

background

The respiratory rate (respiratory rate, RR ), combined with body temperature, blood pressure and pulse, are the four vital signs that are closely related to a person's state of health, such as the progression of a disease and the clinical severity of the disease. RR is a good discriminator in the detection of high-risk patient groups before severe processes, such as cardiopulmonary arrest or admission to the intensive care unit, become an issue. Of course, RR monitoring makes it easier to identify breathing-related symptoms such as sleep apnea, chronic obstructive pulmonary disease, and asthma. They also have changes in the RR also have a strong connection with psychological issues, such as stress and anxiety and cognitive load. RR can also be linked to polysomnography, as its changes reflect different phases of sleep. The current practice for monitoring the RR in hospitals and nursing is usually based on manual observation by clinical staff. This prevents the collection of continuous and reliable data on a scale that allows early risk analysis and early intervention.

Various RR estimation methods have been developed. Overall, the RR estimation methods can be categorized into two groups: contact-based and non-contact-based. In some contact-based methods, sensors are attached near the windpipe or parts of the body (neck, chest, abdomen, etc.), on which the collected signals directly reflect the breathing activities, for example air flow, breathing acoustics, lung movements. Other contact-based methods use sensors, for example PPG and ECG devices, which capture implicit respiratory information that contains a certain type of respiratory modulation to the main component. A conventional and still widely used non-contact method requires trained clinical observations by counting the number of breast lifts during a given period (e.g., a full minute). The non-contact methods use sensors such as pressure, optical imaging, thermal imaging, and radar signals that could detect mechanical movement or heat maps caused by breathing. It is also pointed out that non-contact recording methods offer the patient better comfort and are more precise, since “the stress caused by a contact device can change the respiratory rate”.

Despite a large number of existing detection techniques for RR monitoring, most of the above systems use a single device, each of which requires a trained technician or trained clinician to operate. Manual operation makes this difficult for real-time, large-scale monitoring.

summary

The development of automated and continuous RR monitoring solutions is helping to create more robust and scalable early interventions in challenging healthcare environments, with RR can be a crucial indicator.

The present disclosure provides, inter alia, a method for determining a respiratory rate ( RR ) of a patient, the method comprising the steps of measuring a patient's weight by retrieving weight-related information from a plurality of bed-mounted load sensors, transmitting the weight-related information to a server.

There is also a step of data cleansing, the step of data cleansing comprising: rearranging data frames by comparing time stamps and / or acquiring redundant data by comparing neighboring frames, and or removing the redundant data and / or marking up missing time frames by comparing differences between Time stamps and a total number of frames collected.

Further, there may be a step of non-linear transformation using a sigmoid function to amplify signals caused by small magnitude breathing while suppressing a body movement signal obscuring the respiratory information.

Furthermore, there can be a step of segmentation with overlapping, displaceable rectangular windows that extend over several frames, in particular seven frames, with the step of one frame, further comprising a main component analysis for dimensional reduction.

There may also be a data reliability analysis comprising: applying a binary acceptance rejection classifier to recognize the reliability of each frame.

There can be three different types of classifiers with supervised training taking place offline, with the basic truth being marked manually. The reference identifier of a frame, i.e. whether or not a frame is reliable to contribute to the RR estimation, is marked manually. These reference markings are also known as the basic truth.

In one aspect, a support vector machine (support vector machine, SVM ) be applied.

In a further aspect, a multilayer perceptron (multilayer perceptron, MLP ) be applied.

A hybrid recurrent neural network (hybrid RNN ) be applied.

Data frames that are recognized as reliable can be combined into several fragments of different lengths.

A breathing rate ( RR ) can be extracted using contributions from the fragments while ignoring unreliable and missing frames.

Peak detection can be applied to each fragment, with a weighted combination followed, depending on the length between the first and last detected peak in each fragment (if any) and the fragment length to detect the respiratory rate.

A time-domain Hilbert transform can be applied to each fragment in order to obtain an analysis signal from which the phase information can be calculated and unfolded, further comprising a slope fitting for all fragments, with a shared slope parameter and a further distortion parameter for each segment, where the respiratory rate ( RR ) is mapped from the slope parameter.

In one aspect, the number of frames in each block, the variation in the detected peak-to-location distance and the fitting errors are used as indicators of the level of confidence in the respiratory rate ( RR ) used.

The present disclosure also provides a system for determining a respiratory rate ( RR ) of a patient. The system can comprise a plurality of bed-mounted load sensors (weight sensors), which are arranged essentially at the edge around a bed, electronic circuits for receiving and processing corresponding weight data from the plurality of bed-mounted sensors, a data logger for temporarily storing the weight data, and a transmitter for transmission the weight data to a data processing unit, wherein the data processing unit is configured to classify a slice of respiratory data in the received data, a respiratory rate ( RR ) in the disc and the recorded respiratory rate ( RR ) as a template for capturing additional respiratory rate information.

The central data processing unit can also be configured to provide a SVM , hybrid RNN and or MLP to be used for classification.

The central data processing unit is configured to use a Hilbert transformation.

The present disclosure is based on the knowledge that IoT technologies (Internet of Things) can provide a solution for the restrictions mentioned at the beginning. The present disclosure provides a non-contact IoT-based system for real-time in-bed RR monitoring.

An IoT system could have a complex structure, such as sensor nodes, data storage, network, users. One advantageous concept, however, is quite simple, that is, a "communication actuation network" with an extensive distribution of sensor devices.

The disclosed system consists of five main components: bed sensor devices, gateway, back-end, machine learning ( ML ) Algorithm and user interface ( UI ). Each equipment set contains four load sensors, which are mounted under the feet of the bed, which are connected by wire to a data logger, which calibrates, processes and transmits the measured data. The data packet sent by the data logger contains waveform data associated with a respiration, as well as other metadata, such as the device ID, patient ID, weight, center of gravity and posture. The gateway receives data over a wireless network and creates a common data format before storing it in memory. With various configurations and specifications, the back end coordinates database management and other web services such as message bus, authentication middleware and network protocols. The ML component estimates the real-time RR and the user interface visualizes the data associated with the registered patients. The disclosed system offers the following advantages over conventional RR extraction methods.

The methods and system of the present disclosure have very little operational anomaly because the bed mounted sensors are non-contact and can be placed under the bed for the purpose of monitoring breathing while at rest or asleep. As a result, the person does not feel any disturbance due to the continuous operation. In addition, there are no restrictions on posture or position required, making it extremely convenient for users. The present disclosure also offers easily extensible RR estimation methods. The gateway, back-end and ML component can run on a remote server instead of a physical server at the monitoring point. The ML module can run continuously, but does not interfere with the data collection process. The ML models used can be retrained and fine-tuned to accommodate new patterns and statistics in the data collected.

The present disclosure also provides a scalable and expandable ecosystem. New devices can be added and registered and assigned to assigned patients by authorized users via the user interface (UI). The ML function can be expanded with more features according to customer requirements, for example to predict sleep quality. Other types of devices, for example those that are worn on the body, can be integrated into the system in order to collect physiological signals for comparison or for merging with the respiratory signals. Remote monitoring can also be provided. Authorized users, such as clinical staff, can have access to the user interface and can remotely view and manage both real-time and historical reports. The technical novelties of this work include i) Developing an end-to-end system from collecting raw sensor data to end-user interface and visualization. ii) Providing a flexible IoT architecture that can also contain other sensor devices, which enables great system flexibility. iii) Establishing a novel method for analyzing the noisy and dynamic data for capturing the respiratory rate from the inconspicuously collected sensor data and evaluating the work and comparing the performance with conventional benchmark algorithms.

The disclosed methods and systems are configured to extract an accurate real-time RR from the bed sensor device.

The present disclosure also provides a user interface (UI) configured to operate in accordance with and in conjunction with aspects of this disclosure.

Figure list

• 1 eine vereinfachte schematische Darstellung eines loT-basierten RR-Überwachungssystems gemäß Aspekten der vorliegenden Offenbarung;
• 2 eine weitere vereinfachte schematische Darstellung des RR-Überwachungssystems 1, in dem mehr Details gezeigt sind;
• 3 eine 30 Sekunden lange Sensor-Wellenform-Darstellung;
• 4 eine vereinfachte schematische Darstellung eines MLP-Netzwerks gemäß Aspekten der vorliegenden Offenbarung;
• 5 eine vereinfachte schematische Darstellung eines hybriden RNN-Netzwerks gemäß Aspekten der vorliegenden Offenbarung;
• 6 eine vereinfachte schematische Darstellung, in der die Anwendung von HT auf die Signale in den oberen zwei Zeilen von 3 veranschaulicht wird;
• 7 eine vereinfachte schematische Darstellung, in der eine Konvergenz des MLP- und des RNN-Verfahrens auf den Validations-Datensatz veranschaulicht;
• 8 eine vereinfachte schematische Darstellung, in der ein Vergleich von Vorhersageergebnissen (vor dem Vorzeichen-Operator) aus den drei trainierten Modellen mit der Grundwahrheit veranschaulicht wird;
• 9 eine vereinfachte schematische Darstellung, in der die Anwendung von sowohl Spitzenzählung als auch HT auf die rohe Wellenform veranschaulicht wird, um für jede Minute eine RR zu extrahieren;
• 10 eine vereinfachte schematische Darstellung, in der die Kombination des SVM-Modells mit der RR-Schätzung für eine robustere Atemfrequenzextraktion veranschaulicht wird; und
• 11 eine vereinfachte schematische Darstellung, in der die Kombination des hybriden RNN-Modells mit der RR-Schätzung veranschaulicht wird.

Further aspects and characteristics of the present disclosure will become apparent from the following description of the embodiments with reference to the accompanying drawings. It shows:

• 1 a simplified schematic representation of a loT-based RR monitoring system in accordance with aspects of the present disclosure;
• 2 another simplified schematic representation of the RR monitoring system 1 where more details are shown;
• 3 a 30 second sensor waveform display;
• 4th a simplified schematic representation of an MLP network in accordance with aspects of the present disclosure;
• 5 a simplified schematic representation of a hybrid RNN network in accordance with aspects of the present disclosure;
• 6th a simplified schematic showing the application of HT to the signals in the top two lines of 3 is illustrated;
• 7th a simplified schematic representation in which a convergence of the MLP and the RNN method on the validation data set illustrates;
• 8th a simplified schematic representation in which a comparison of prediction results (before the sign operator) from the three trained models with the basic truth is illustrated;
• 9 is a simplified schematic showing the use of both peak counting and HT on the raw waveform is illustrated to make one for every minute RR to extract;
• 10 a simplified schematic diagram illustrating the combination of the SVM model with the RR estimate for a more robust respiratory rate extraction; and
• 11th a simplified schematic diagram illustrating the combination of the hybrid RNN model with the RR estimate.

Detailed description of embodiments

1 is a simplified schematic of an IoT-based RR monitoring system 1 in accordance with aspects of the present disclosure. There is a bed sensor unit 2 and its wired connection with the data logger 3 (above). The data logger 3 calibrates, preprocesses and sends out data measurements. A signal snippet that was output by the bed sensor for over 15 minutes 2 takes, is plotted (below). The embedded plot enlarges a short segment that is highlighted in the small rectangle when the patient is not moving significantly, from which a periodic breathing pattern is observed. The A-axis shows the magnitude A. and the T-axis the time in the format MM: SS.

A bed sensor device 9 contains a data logger 3 and four sensor units 2 which can essentially be configured as force sensors (ie load cells). The sensor units 2 can show a high level of sensitivity. In this way, any pressure change due to mechanical movements can be recorded and reflected in the vibrations of the sensor measurement, including the very trivial breathability. The data logger 3 calibrates the raw data and generates a waveform signal that reflects body movements. Other information, such as body weight, center of gravity and posture, can also be obtained from the data logger 3 extracted. This additional information is not included in the scope of the present disclosure and will not be discussed further. The wireless data transmitter and receiver (transceiver) 10 is in the data logger 3 integrated for data communication. A bed mounted device and its output waveform signal are in 1 illustrated.

The bed sensor system 9 detects vibrations caused by mechanical movements. Regular movements, such as turning around, sitting up, coughing, result in a relatively large magnitude A. in the measured waveform, which obscures the respiratory information, as from the tenth minute in 1 shown (below). However, when the participant / patient is resting, breathing-related movements can be detected by the measured waveform, although they are only of a very small magnitude A. (two orders of magnitude smaller compared to that caused by regular movement). However, these “weak” signals contain rich respiratory information, from which the underlying respiratory information can be determined using appropriate signal processing techniques RR can be appreciated.

2 Figure 3 is another simplified schematic of the RR monitoring system 1 , with more details shown. Various components, that is, local devices 2 , 3 who collect patient biological information, service providers (servers) 4th , Consumer 5 , can either be distributed or share the same access point. The schematic illustrates a more difficult distributed case in which the local devices 2 , 3 and the server 4th via a VPN (Virtual Private Network) 6th are connected while having authorized access between the server 4th and the end users 5 is established to ensure data security and data protection for patients.

In other words shows 2 a potential general structure of the system 1 , using a scalable architecture that enables real-time monitoring of multiple participants. Devices distributed in different locations 11th are with the central data processing point 12th that houses most of the compute-intensive and resource-hungry services, including the gateway 13th , Back-front ends 14th and the ML component 15th .

This central data processing point 12th here also as the "server" 12th are designated.

A lot of back end services are running on the server 12th to ensure the efficiency, safety, stability and thus high quality user experiences. For example, these are authentication services for managing user accounts, device and patient services for registration and assignment, output, comment and notification services for reporting problems and for pushing a front-end visualization. To support compatibility with other health and care platforms, the data packets containing the collected data are converted into FHIR messages (Fast Healthcare Interoperability Resources). A RabbitMQ Message Broker organizes the efficient handling of the large amount of data packets collected by the gateway from the distributed devices 8th queues the converted FHIR messages and forwards them to the storage system. MongoDB 16 is used to manage the database which contains several security measures related to data protection.

The front-end component 17th enables (authorizes) users to access an interactive web interface to i) manage user accounts, ii) provide real-time monitoring and also access historical data, iii) manage the device and patient reports, e.g. add new patients and register iv) issue warnings when the system detects irregular patterns in participant measurements that may indicate activity and / or health risks.

The processing and storage capacity of the system 1 can be increased by cloud computing and big data analysis techniques on a cloud server, which could significantly improve the scalability of the system, thereby enhancing its possible use in the wards and hospitals with a large patient population. This also reduces the hardware costs for creating and maintaining local physical servers for individual use in home environments, for example for care at home for elderly people living alone. In addition, a cloud server enables system maintenance (e.g. service upgrade) and maximizes stability (e.g. because problems with the compatibility of a local server are avoided).

After the data collection and storage components have been deployed, the data is analyzed using analysis and ML methods. Different ML methods have been applied to different cases in this field. However, the existing techniques are not scalable for the dynamic and multivariate data in the system. There are methods for RR estimation which use the dynamic sensor data in the system and then in the evaluation section. The performance of the system is compared with the conventional benchmark algorithms.

3 is a representation of a 30 second sensor waveform with the T-axis showing the time in seconds that the participant goes through four stages with (lower three) and without (uppermost) body movements other than breathing. Body movements modify and obscure breathing activity in the waveform, resulting in unreliable data for the RR estimation.

The bed sensor 2 creates a waveform related to breathing when the patient is at rest or nearly motionless. However, it is not practical to keep a patient still for a long time. Therefore, the influence of outliers and unreliable data caused by body movements is mitigated while the RR is continuously monitored.

3 illustrates the bed sensor signals collected during the participant 4th Stages going through: immobile in a reclined position 19th , speaking in a reclined position 20th , randomly moving 21 and panting 22nd (the figures are shown from top to bottom). When the participant lies motionless 19th , the signal produces a sinusoidal pattern with a very small magnitude A. . Each transition from a maximum (peak) to a minimum is assigned to a full breathing cycle in this case. the RR can then be calculated directly by counting peaks (maxima) within a certain period of time.

If there is other activity besides breathing, such as speaking 20th , the waveforms associated with breathing collected by the sensors can be significantly distorted. Speech-breathing has irregular cycles due to changes in the depth and duration of inhalation and exhalation. When speaking, muscles of the lower jaw, neck, and shoulder are activated, and there are chest movements, and these kinematics are not negligible compared to breathing activity, introducing artifacts and outliers. If "heavier" movements occur, such as random movements 21 and panting 22nd , become large magnitudes A. as in the last two subplots in 3 is shown. These movements show much more complex and irregular patterns due to their randomness in position, direction and strength of movement. Most importantly, these movements distort the breathing information. Therefore, the affected sensor measurements for the RR estimation may not be reliable.

One solution to this data reliability problem is to use a classifier followed by RR estimation. The classifier recognizes the reliability of a collection or a short segment of the sensor measurement, i.e. it is a binary acceptance-rejection classifier. Information contributed by the selected reliable data can then be used to robustly calculate the respiratory rate.

Body movements often cause larger magnitudes A. (greater energy) than breathing activities. A simple solution is to impose an energy threshold on the waveform signal. However, this solution can be difficult to adjust to detecting patterns caused by breathing and very small movements, such as speaking. In addition, it is difficult to define the threshold value when considering individual differences in weight, position (edge or center of the bed), posture (lying on your back, lying on your side) and breathing type (deep breathing, light breathing ), all of which have an impact on the energy level of the signal.

An advantageous method according to the present disclosure is a supervised training by supplying sufficiently labeled data and a corresponding classification model. A supervised algorithm can be used to uncover the hidden structure and explore the underlying information contained in the data.

A large number of monitored classification methods can use binary clustering, for example the models logistics regression ( LR Decision trees, random forest, support vector machine SVM ) and deep neural network ( DNN ) execute. Decision trees and random forests are often used on data with multiple types of characteristics. However, this is not the case with a time-series one-dimensional waveform. LR has a tendency to overfitting and assumes a linear combination of the entered characteristics. On the other hand shows SVM Advantages of a rich hypothesis space with a large number of kernel functions that handle linearly challenging data. Models with neural networks are able to “discover a delicate structure in large data sets” if a sufficiently large set of training data is available. In accordance with the present disclosure SVM and DNN intended to build a robust classifier.

Let s (t) be the waveform signal indexed from time t, and _{Let -F s} be the sample rate. The objective is to build a classifier that records the reliability of the nth frame s (n) = [s ((n - 1) F _s + 1), ..., s (nF _s )] ^T , with the superscript T represents the transposition. This basic truth reliability label is called y (n) ∈ {0,1}. To construct the inputs to the disclosed classification models, the following features are extracted.

wobei σ der Skalierungsfaktor ist. Der normalisierte n - te Rahmen wird als s̅(n) bezeichnet.As already mentioned, breathing activities and other body movements result in different energy levels. To suppress unreliable outliers with large magnitudes A. and to emphasize the stable, but "weak" breath waveform, the signal is first normalized to the range of (-0.5, 0.5) with a scaled and shifted sigmoid function: $$\overline{s}\left(t\right)=1/\left(1+\text{exp}\left(-\sigma s\left(t\right)\right)\right)-\mathrm{0.5,}$$

where σ is the scaling factor. The normalized nth frame is denoted as s̅ (n).

To use the temporal information beyond the current frame n, e.g. the correlation between neighboring 2L + 1 frames, frames centered over the nth frame are extracted and concatenated, with 2L + 1F _s dimensional data resulting: $$\text{z}\left(n\right)={\left[\overline{\text{s}}{\left(n-\mathrm{L.}\right)}^T,...,\overline{\text{s}}{\left(n+\mathrm{L.}\right)}^T\right]}^T.$$

wobei W E ℝ^{Qx(2L+1)F s} Q Eigenvektoren enthält, die den größten Q Eigenwerten entsprechen, überspannt von dem Eigenraum {z(n)}, n = 1, 2, ... , N von allen N Trainingsabtastungen.To avoid excessive fitting of high-dimensional data, a principal component analysis ( PCA ) is applied to reduce the data dimension. The PCA method generates the feature x (n): $$\text{x}\left(n\right)=\text{Wz}\left(n\right),$$

where WE ℝ ^{Qx (2L + 1) F s} Q contains eigenvectors corresponding to the largest Q eigenvalues spanned by the eigenspace {z (n)}, n = 1, 2, ..., N of all N training samples.

In principle, other summary techniques that represent the time series efficiently can also be used for feature extraction. For example, the discrete cosine transformation and eigenvalue characterization of the segmented data can also be used for dimension reduction. In the present embodiment, however, the objective is to implement a (near) real-time method and avoid the use of methods that have a higher computational complexity. The extracted features retain and represent the key characteristics of the data and allow a more robust classifier to be built.

wobei || · | die Frobenius-Norm ist, C der Box Constraint ist, der den Penalty-Ausdruck von Schlupfvariablen ξ_n gewichtet; ϕ(·) eine Konvertierung der eingegebenen Daten in einen höherdimensionalen Raum anwendet.2) SVM : The extracted features are fed into the SVM training process using the c-support vector classification which aims to optimize: $$\begin{array}{l}\text{bad}\underset{\text{p},p_0,\xi }{\text{min}}=0.5{\Vert \text{p}\Vert }^2+\mathrm{C.}{\displaystyle \sum _{n=1}^N{\xi }_n}\\ \text{s}\text{.t}\text{.}y\left(n\right)\left({\text{p}}^T\varphi \left(\text{x}\left(n\right)\right)+p_0\right)\ge 1-{\xi }_n\cdot {\xi }_n>=\mathrm{0,}\end{array}$$

where || · | is the Frobenius norm, C is the box constraint that weights _{the penalty expression of slack variables ξ n;} ϕ (·) applies a conversion of the entered data to a higher-dimensional space.

Equation (4) is solved by a dual optimization problem defined on kernel space, where the kernel is the inner product of two converted variables. The kernel can be selected from a number of kernel functions. The kernel of the radial basis function (RBF) was selected based on the pre-classification tests. The dual optimization process was made possible by the box constraint C. , and a kernel scaling parameter γ, which can be set via a heuristic procedure such as a grid search, is regulated after the kernel function has been selected. The trained modelψ ^(svm) (·) is applied to a data point to measure reliability: $$\widehat{y}\left(n\right)=\text{sign}\left({\mathrm{\Psi }}^{\left(\text{svm}\right)}\left(\text{x}\left(n\right)\right)\right).$$

über eine K-fache Kreuzvalidation trainiert werden, wird die letzte Vorhersage zu: $$\widehat{y}\left(n\right)=\text{sign}\left({\displaystyle \sum _{k=1}^K{\mathrm{\Psi }}_k^{\left(\text{svm}\right)}}\left(\text{x}\left(n\right)\right)\right).$$

When ensembles of SVM models obtained by bagging ${\mathrm{\Psi }}_k^{\left(\text{svm}\right)}(\cdot ),k=\mathrm{1,}...,K$

are trained via a K-fold cross-validation, the last prediction becomes: $$\widehat{y}\left(n\right)=\text{sign}\left({\displaystyle \sum _{k=1}^K{\mathrm{\Psi }}_k^{\left(\text{svm}\right)}}\left(\text{x}\left(n\right)\right)\right).$$

3) DNN : DNN can learn the complex underlying structures in the data. In the present embodiment, a conventional multilayer perceptron (MLP) model is first implemented to construct the data reliability classifier.

4th FIG. 3 is a simplified schematic representation of an MLP network in accordance with aspects of the present disclosure. The implemented MLP network contains three hidden layers, each from a dropout layer 23 to be followed. Black lines between the input layer and the first hidden layer indicate the dense combination 24 , where the bias term is added over a unit of 1. The following layers and their dense connections 24 have been simplified for illustration purposes. Nonlinear neurons are after the fully connected (or dense) layer 24 used, e.g. rectified linear units ( ReLU ) and sigmoid functions plotted above.

MLP, also known as the Vanilla Network, has a simple feedforward structure, as in 4th shown. Units in two adjacent layers are via fully connected dense links 24 connected, followed by a non-linear activation function. In one embodiment ReLU used on hidden layers for fast computation in backpropagation and fast convergence. The sigmoid neuron is used on the final output layer to narrow the prediction within the range of (0,1). To avoid overfitting, it is done after each dense layer 24 a dropout layer 23 is used, which causes units to be dropped randomly along with their connections in a predefined ratio.

The same input feature for the SVM classifier, i.e. the PCA component from 2L + 1 concatenated frames in equation (3), is ^{inserted into the MLP model ψ (mlp)} (·) to obtain the final prediction: $$\widehat{y}\left(ñ\right)=\text{sign}\left({\mathrm{\Psi }}^{\left(\text{mlp}\right)}\left(\text{x}\left(n\right)\right)-0.5\right).$$

wobei zur größeren Einfachheit der Zeitindex n weggelassen wird.To train the MLP model ψ ^(mlp) (), the cross entropy is minimized as the loss function during backpropagation: $$-y\text{log}\left({\mathrm{\Psi }}^{\left(\text{mlp}\right)}\left(\text{x}\right)\right)-\left(1-y\right)\text{log}\left(1-{\mathrm{\Psi }}^{\left(\text{mlp}\right)}\left(\text{x}\right)\right),$$

where the time index n is omitted for the sake of simplicity.

However, the simple MLP structure has the restriction that its recording field is restricted by the fixed input size, e.g. 2L + 1 frame. Therefore the long time information is not used. To compensate for the neglect of long-term context information, another hybrid network was created, mainly based on the recurrent neural networks ( RNN ) implemented.

5 is a simplified schematic representation of a hybrid RNN in accordance with aspects of the present disclosure. The implemented hybrid RNN contains two stacked layers of long-lasting short-term memory ( LSTM , Long Short-Term Memory) and a backward LSTM that uses future L frames to create hidden forward 26th and backward tensors 27 that are merged by summation. A fully connected dense layer with a sigmoid activity function is applied at each point in time of the fused tensor to generate the successive predictions. The LSTM layers are illustrated with thick hollow arrows, and the blocks 25th are the tensor outputs. RNN can take inputs of various lengths of T, and one gets T ≥ 2L ≥ 2L + 1 frame of input data. Instead of using the extracted features, such as the PCA component, included in the process SVM and MLP are used, normalized time series s̅ (n), n = 1, ..., T are used directly. RNN layers with LSTM units are used which use the history information from all previous frames when predicting the current frame. An alternative is bidirectional LSTM ( BiLSTM ) exploring both the previous context and the future context to learn the time dependencies for the current frame. BiLSTM however, introduces a latency problem and requires an entire sequence to be traversed, which is not suitable for on-line processing. To solve this problem, a hybrid structure was used as described in 5 is shown. The disclosed model modifies the mechanism of bidirectional recurrent networks, while at the same time allowing on-line processing with acceptable latency. To provide a consistent basis for comparison with the models SVM and MLP using both previous and future L frames, the disclosed RNN structure also has L frame latency.

The normalized time series [s̅ (1), ..., s̅ (T - L)] first goes through two stacked LSTM layers, both of which contain D filters, which results in a forward tensor 26 of size D × (T - L ) results. In parallel, the time series [s̅ (L + 1), ..., s̅ (T)] are subdivided into T - 2L overlapping segments that each contain L + 1 frames. Each segment goes through a backward LSTM layer containing D filters and only the last frame output is reserved. Outputs from all segments are concatenated using a backward tensor 27 of a size D × (T - 2L). The forward 26 and the backward tensor 27 are added together, and to maintain dimensional consistency, only the last (T-2L) frames of the forward tensor are added 26th with the backward tensor 27 united. Sigmoid units are applied to the pooled tensor at each frame, producing T-2L label estimates associated with T-2L frames indexed from L + 1 to T-L.

To apply this RNN Network on an online prediction scenario (ie generation of a label prediction of when a new frame will arrive), a delay of L seconds is allowed in order to allow the future L frames to update the backward tensor 27 to obtain. In addition, hidden outputs of the two stacked LSTM layers are kept on the previous frame, which represent the history information. If one denotes the RNN model as ψ ^(mn) (), the reliability is obtained as: $$\widehat{y}\left(n\right)=\text{sign}\left({\mathrm{\Psi }}^{\left(\text{mn}\right)}\left(\left[\overline{\text{s}}\left(1\right),...,\overline{\text{s}}\left(n+\mathrm{L.}\right)\right]\right)-0.5\right).$$

After applying the converged classification models to the collected waveform to detect whether or not a frame of data is reliable, the RR extracted from these reliable frames. Two RR estimation methods are implemented, one on a peak counting and the other on a Hilbert transform ( HT ) based.

From the top line in 3 it can be observed that a complete breathing cycle has a maximum and a minimum. As a result, the number of breaths in a given period is equal to the count of the maxima (peaks).

Assume I segments of consecutive data are recorded as reliable during a period of time, for example over a minute, and t _{i, 1} to t _{l, 2 is} the starting index or the end index of the i-th segment. This segment is denoted _{by s i} = [s (t _{i, 1} ), ..., s (t _{i, 2} )] ^T. Peaks can be recorded as the local maxima in the curve. To avoid false positives, restrictions, such as a threshold value for the magnitude, should be considered A. and imposing an altitude difference threshold with the adjacent minimum. Assume that there are a total of J _i maxima in s _i , arranged at p _{i, j} , j = 1, ..., J _i . captured, then can RR can be calculated as follows: $$\text{RR}=\frac{\left({\displaystyle \sum _{i=\mathrm{1,}{J}_i>1}^{\mathrm{I.}}\left(J_i-1\right)+{\displaystyle \sum _{i=\mathrm{1,}{J}_i\le 1}^{\mathrm{I.}}{J}_i}}\right)60/\mathrm{F.}s}{{\displaystyle \sum _{i=\mathrm{1,}{J}_i>1}^{\mathrm{I.}}\left(p_i,J_i-p_{i\mathrm{,1}}\right)+{\displaystyle \sum _{i=\mathrm{1,}{J}_i\le 1}^{\mathrm{I.}}\left(t_{i\mathrm{,\; 2}}-t_{i\mathrm{,1}}\right)}}}.$$

6th is a simplified schematic illustrating the application of HT to the signals in the top two lines of 3 is illustrated. The application of HT to the signals in the top two lines of 3 is advantageous for obtaining the folded phase B (above) from which the IF is calculated and based on RR can be shown (below). Using the HT-based method of slope fitting, the RR values of 20 and 16 were calculated for these two sequences.

An HT method can be used to calculate instantaneous frequency (IF), which is the slope of the unfolded phase of the analysis signal. $$\text{IF}=\nabla \text{U}\left(\angle H(\cdot )\right),$$

In the present embodiment, HT to the signals in the top two lines of 3 applied to get the folded phase B (above), for which the IF is calculated and based on RR can be shown (below). Using the HT-based method of slope adjustment, the RR values of 20 and 16 were calculated for these two sequences, respectively.

If H (·) HT then U (·) unfolds the phase, and ∇ computes the gradients. For the present embodiment, the IF can directly refer to the RR are represented by: $$\text{RR}=\frac{60}{2\pi }\text{IF}\text{.}$$

dann können die Beiträge zur Steigungsanpassung aus allen I Segmenten wie folgt formuliert werden: $$\underset{\eta }{\underbrace{\left[\begin{array}{c}\begin{array}{l}{\mathrm{\alpha }}_1\{\begin{array}{l}\alpha \left(t_{\mathrm{1,1}}\right)\\ \alpha \left(t_{\mathrm{1,1}}+1\right)\\ ⋮\\ \alpha \left(t_{\mathrm{1,2}}\right)\end{array}\\ ⋮\\ ⋮\end{array}\\ \mathrm{\alpha }I\{\begin{array}{l}\alpha \left(t_{I\mathrm{,1}}\right)\\ \alpha \left(t_{I\mathrm{,1}}+1\right)\\ ⋮\\ \alpha \left(t_{I\mathrm{,2}}\right)\end{array}\end{array}\right]}}\approx \underset{\mathrm{\Phi }}{\underbrace{\left[\begin{array}{ccccc}1& 1& 0& \dots & 0\\ 2& 1& 0& \dots & 0\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \Delta t_1& 1& 0& \dots & 0\\ 1& 0& 1& 0& \dots \\ 2& 0& 1& 0& \dots \\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& 0& \dots & 0& 1\\ 2& 0& \dots & 0& 1\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \Delta t_I& 0& \dots & 0& 1\end{array}\right]}}\underset{\text{b}}{\underbrace{\left[\begin{array}{c}{b}_0\\ b_1\\ ⋮\\ b_I\end{array}\right]}},$$

wobei Δt_I = t_i,2 - t_i,1 + 1 ist und b die Gesamtsteigung b₀ und einen Bias-Ausdruck b_i für jedes i - te Segment enthält, dessen Lösung in geschlossener Form wie folgt ist: $$\text{b}={\left({\mathrm{\Phi }}^T\mathrm{\Phi }\right)}^{-1}{\mathrm{\Phi }}^T\eta .$$

However, the IF is very sensitive to noise. A person's breathing pattern is not an ideal sine wave. Local changes in inhalation and exhalation would therefore make large changes in the recorded RR cause, as in 6th What is illustrated is where the dotted line shows the recorded signal of a person lying down who is completely immobile, and the solid line shows the recorded signal of a person lying down who is speaking. The IF is essentially the slope of the phase curve at each point in time. However, if one observes the overall slope, it is relatively smooth and reflects more stable breath information. As a result, a robust RR value can be obtained from all reliable data points through an adaptation by fragmented line fitting. α _i is the unfolded phase from H (s _i ): $${\alpha }_{\text{i}}=\text{U}\left(\angle H\left({\text{s}}_i\right)\right)={\left[\alpha \left(t_{i\mathrm{,1}}\right),\cdots ,\alpha \left(t_{i\mathrm{,\; 2}}\right)\right]}^T,$$

then the contributions to the slope adjustment from all I segments can be formulated as follows: $$\underset{\eta }{\underbrace{\left[\begin{array}{c}\begin{array}{l}{\mathrm{\alpha }}_1\{\begin{array}{l}\alpha \left(t_{1.1}\right)\\ \alpha \left(t_{1.1}+1\right)\\ ⋮\\ \alpha \left(t_{1.2}\right)\end{array}\\ ⋮\\ ⋮\end{array}\\ \mathrm{\alpha }\mathrm{I.}\{\begin{array}{l}\alpha \left(t_{\mathrm{I.}\mathrm{,1}}\right)\\ \alpha \left(t_{\mathrm{I.}\mathrm{,1}}+1\right)\\ ⋮\\ \alpha \left(t_{\mathrm{I.}\mathrm{,\; 2}}\right)\end{array}\end{array}\right]}}\approx \underset{\mathrm{\Phi }}{\underbrace{\left[\begin{array}{ccccc}1& 1& 0& ...& 0\\ 2& 1& 0& ...& 0\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \Delta t_1& 1& 0& ...& 0\\ 1& 0& 1& 0& ...\\ 2& 0& 1& 0& ...\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& 0& ...& 0& 1\\ 2& 0& ...& 0& 1\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \Delta t_{\mathrm{I.}}& 0& ...& 0& 1\end{array}\right]}}\underset{\text{b}}{\underbrace{\left[\begin{array}{c}{b}_0\\ b_1\\ ⋮\\ b_{\mathrm{I.}}\end{array}\right]}},$$

where Δt _I = t _{i, 2} - t _{i, 1} + 1 and b contains the total slope b ₀ and a bias expression b _i for every i - th segment, the solution of which in closed form is as follows: $$\text{b}={\left({\mathrm{\Phi }}^T\mathrm{\Phi }\right)}^{-1}{\mathrm{\Phi }}^T\eta .$$

erhalten.By integrating the sampling frequency, which is omitted from equation (14), using equation (12) $\text{RR}=\frac{60{\mathrm{F.}}_s}{2\mathrm{<figure-callout\; id="0"\; label="\pi \; b"\; filenames="DE102020117244A1\_0023.png,DE102020117244A1\_0025.png"\; state="\{\{state\}\}">\pi </figure-callout>}}{b}_{}{}_0$

receive.

To train and validate the disclosed ML methods for RR estimation, data was recorded from four participants, each lasting approximately one hour. The data were recorded on a standard single bed as shown in 1 is shown. A trained observer counted the number of breaths during set periods of time when breathing activity was visible. The observational information was used as a ground truth RR to validate the disclosed method. A total of 56 minutes were successful with one basic truth RR logged. Information about the four participants, including gender, age, weight, and body mass index (BMI) is summarized in Table I. No one had any known pre-existing respiratory illnesses that would impede ventilation.

TABLE I: Summary of Participant Demographic Information # gender old Weight [kg] BMI 1 M. 25th 93 24.26 2 F. 23 50 19.53 3 M. 46 68 21.4 4th F. 25th 62 21.9

To simulate breathing activities, as well as other activities and factors that could disrupt the breathing pattern, all participants were asked to perform the following 17 actions, each lasting 2 minutes: lying on their back, lying on their stomach, lying on the left side of the body, lying on the right side of the body, lying on his back position near the right edge of the bed, lying on his back position near the left edge of the bed, random movements on the bed, legs tossing, coughing, deep Breaths, wheezing (gasping for breath), speaking, sitting up in the middle of the bed, sitting on the edge of the bed, quick shallow breaths, body tremors, leg movements.

• 1) SVM: Ein SVM-Klassifikator wurde zuerst trainiert. Zum Extrahieren von in dem SVM-Modell verwendeten Merkmalen wurde der Skalierungsfaktor σ in Gleichung (12) empirisch auf 10 gesetzt. Verschiedene Zeitparameter L wurden getestet. PCA wurde angewendet, um die größten Eigenwerte zu erhalten, die über 95% Eigenraum beherrschen. Insgesamt wurden 14078 - 8L Merkmalsvektoren erhalten, die zum Testen des Trainings (80% - 20%) aufgespalten wurden. Eine 10-fache (K = 10) Validation wurde angewendet, um ein Ensemble von SVM-Modellen ${\psi }_k^{\left(svm\right)}(\cdot ),k=\mathrm{1,}\dots ,K$
  
  zu generieren.

A total of 14078 seconds of data was recorded. To train the reliability classifiers mentioned in Section III, the data reliability per second (frame) was marked. Three classifiers were trained as detailed below.

• 1) SVM : An SVM classifier was trained first. To extract features used in the SVM model, the scaling factor σ in equation (12) was empirically set to 10. Various time parameters L were tested. PCA was used to get the largest eigenvalues that dominate over 95% eigenspace. A total of 14078-8L feature vectors were obtained which were broken down to test the training (80% -20%). A 10-fold (K = 10) validation was applied to an ensemble of SVM models ${\psi }_k^{\left(svm\right)}(\cdot ),k=\mathrm{1,}...,K$
  
  to generate.

The results performed better in terms of prediction accuracy using a Gaussian kernel (i.e., RBF kernel) and were therefore chosen accordingly. This also agrees with an intuition about the RR data that appears to have a multivariate Gaussian distribution. The parameters γ and C were selected via a grid search that varied with the time parameter L.

und Trefferquote $\text{recall}=\frac{\#\text{tp}}{\#\text{tp}+\#\text{fn}}.$

Five candidates of L were tested and performance was evaluated on the test data set using the following three metrics: overall detection error (error), precision $\text{precision}=\frac{\#\text{tp}}{\#\text{tp}+\#\text{fp}},$

and hit rate $\text{recall}=\frac{\#\text{tp}}{\#\text{tp}+\#\text{fn}}.$

These results are summarized in Table II.

TABLE II: quantitative evaluations of the created SVM models L. parameter Evaluations Q C. γ error precision Hit rate 0 3 2 0.05 0.23 0.65 0.71 1 7th 2 0.2 0.14 0.80 0.78 2 11th 2 0.3 0.12 0.83 0.79 3 15th 2 0.4 0.11 0.87 0.78 4th 19th 2 0.5 0.10 0.88 0.79 5 23 2 0.6 0.11 0.88 0.77

As can be seen from Table II, the performance improves as the length of time increases and remains approximately stable after a specific length, for example 7 frames (L = 3). However, a capital L also leads to greater computational complexity due to the relatively high-dimensional data and also a large number of support vectors in the model, which means that it tends to overfitting. It was therefore decided to choose the model trained with L = 3, as highlighted in Table II. To make the comparisons fair, the same length of time was also used for the DNN based models are used.

2) DNN : Two DNN topologies have been implemented: Structures from MLP and hybrid RNN .

The same characteristics that were used for SVM training were also used for the MLP model. As in 4th As illustrated, the MLP model has three hidden layers. The size was set to 100 for each of the fully connected layers, followed by a dropout layer of 0.2. The regularization expression l ₂ was added with a weight of 10 ^-6 . 20% of the training data was used for validation. The learning rate was initialized to 0.001, with a decay rate of 0.99 after each epoch. The Adam optimizer was used in backpropagation. Mini-batches of size 64 were used, with ∈ ℝ ^{64 × Q} for each mini-batch, where Q = 15 for I = 3 situations. A total of 1000 epochs were run and the training data were randomized after each epoch.

for RNN the normalized raw data was used directly instead of the features extracted via the PCA representation in equation (3). As in 5 As shown, two layers of stacked LSTM layers were used, each layer 50 in size, which corresponds to a forward tensor for each point in time 26th a dimension D = 50. The backward LSTM was also 50 in size to maintain consistency. The initial L = 3 frames of the forward tensor 26th were removed before adding with the backward tensor 27 was carried out. Mini batches of one size 32 were used, with each mini-batch ∈ ^{32 × F s × T} , where 20 T 30. The rest of the parameter setup is consistent with the MLP model.

7th is a simplified schematic showing a convergence C. The MLP and RNN methods on the validation data set illustrate where the MLP error 28 , the MLP loss 29 , the hybrid RNN bug 30th and the hybrid RNN loss 31 are shown. The hybrid RNN model did not converge as quickly as MLP . But it showed opposite MLP after 100 epochs E. with a more stable and low loss, ultimately better performance. In addition, the hybrid RNN model also consistently yielded greater accuracy, that is, a lower error rate than MLP . 7th shows the convergence of the two DNN-based models on the valiation data with regard to the objective loss function (cross entropy + regularization expressions) as well as the overall forecast error. It was observed that MLP after 20 epochs E. converged quickly. After that, its loss worsened slightly and then stayed smooth. RNN on the other hand, showed a relatively slow but steady improvement in both loss and accuracy, making it better than MLP showed. The RNN model converged after about 400 epochs E. .

Applied to the test sequences showed MLP an error of 0.10, an accuracy of 0.88, and a hit rate of 0.82, which are marginally better than SVM in terms of accuracy and precision, as well as a relatively larger improvement of 0.04 in hit rate. The RNN model showed benefits with a much larger margin, gave an error of 0.03, a precision of 0.98, and a hit rate of 0.94. RNN had much greater accuracy on test sequences than the trained data set, for the following reasons.

In the RNN training phase, the forward time information was only used for short sequences that lasted 20 T 30 frames. The first few frames were more likely to suffer from a higher detection error due to the lack of long time dependencies. At the inference level, the trained RNN model was applied to much longer sequences, with most frames (except the first few frames) benefited from the accumulated time dependencies, resulting in greater accuracy.

To demonstrate the difference in the performance of the three reliability estimation models, we provided an example of applying these models to a sequence lasting one hour, with the direct prediction results shown before the sign operator. Overall showed SVM and MLP performed similarly, and both suffered from insecurities, particularly from the 30th to the 45th minute. The hybrid RNN however, had a higher performance that was also close to the basic truth. This observation was also consistent with the quantitative evaluations also disclosed here.

8th is a simplified schematic representation in which a comparison of prediction results (before the sign operator) from the three trained models with the basic truth 32 is illustrated. The SVM prediction results have been mapped linearly to a range of [0 1] to be consistent with other methods, where 0 is unreliable and 1 is reliable.

9 is a simplified schematic showing the application of both peak counting 33 as well as the HT 34 on the raw waveform (black curve with the left Y-axis) RR is shown to RR extract A for every minute (diamond and circle curves with the right Y-axis). Based on the expert observations during the recording process, the participant was at rest with occasional movements for the first 20 minutes, and the average RR was about 16 breaths per minute. After that, the participant began to move more and more, including taking deep, slow breaths around the 38th minute and quick, shallow breaths around the 53rd minute, both of which lasted 2 minutes. Affected by outliers, the direct RR estimate suffers from large minute-to-minute variations, especially for the peak counting method 33 that is more likely to be overestimated. The HT process 34 suffers from low energy zero crossing noise as illustrated by the highlighted spike around the 11th minute. The short cyan bars are the human observations of the basic truth.

If the RR estimate either has a peak count 33 or one HT 34 is applied directly to the raw waveform, accuracy could be affected by higher noise levels. 9 illustrates the RR extracted directly from a sequence that lasted an hour that was updated every minute. Because of the large amount of unreliable data caused by body movement, the peak counting technique suffers 33 and the HT 34 both from high errors, as is reflected by the large jumps between two adjacent minutes. A restriction has been introduced that a peak is only considered valid if it falls on both sides by at least the standard deviation of each one-minute segment. It still turned out to be the top count 33 suffered more from outliers and showed a trend of overestimation. the HT 34 was relatively more robust than the top count 33 . However, there was also a higher level of detection errors when using the HT method 34 . Mainly the procedure was the HT 34 more likely to be affected by the low energy zero crossing noise, as shown by the spike around the 11th minute (highlighted by the dashed line 35 ) in 9 is shown.

integriert. Die Reliabilitäts-Kennzeichnung wurde zunächst für jeden Rahmen (Sekunde) vorhergesagt. Segmente, die aufeinanderfolgende zuverlässige Rahmen enthielten, wurden dann extrahiert, und diejenigen, die nur ein oder zwei Sekunden lang dauerten, wurden entfernt. Diese Entfernung der Rahmen erfolgte aus folgenden Gründen: i) es ist sehr wahrscheinlich, dass während dieser kurzen Schnipsel keine Spitze erfasst wird, und ii) die Steigung nach der HT variiert für sehr kurze Segmente erheblich. Wenn die Gesamtlänge der verfeinerten Segmente länger als 9 Sekunden (15 Prozent der aktuellen Minute) war oder wenn das längste gültige Segment länger als 6 Sekunden (10 Prozent) dauerte, wurde durch Anwendung von Gleichung (10) und Gleichung (14) eine RR-Schätzung durchgeführt. Ergebnisse an denselben Daten wie 9 sind in 10 dargestellt.To solve this problem, the reliability models set up in Section IV-B were combined with the RR estimation models. First, the trained ensembles were the SVM models ${\mathrm{\Psi }}_k^{\left(\text{svm}\right)}(·),k=\mathrm{1,}...\mathrm{,\; 10}$

integrated. The reliability label was initially predicted for each frame (second). Segments containing consecutive reliable frames were then extracted and those that lasted only a second or two were removed. This removal of the frames was done for the following reasons: i) it is very likely that no peak will be detected during these short snippets, and ii) the slope after the HT varies considerably for very short segments. If the total length of the refined segments is longer than 9 seconds ( 15th Percent of current minute) or when the longest valid segment lasted longer than 6 seconds (10 percent), an RR estimate was performed using equation (10) and equation (14). Results on the same dates as 9 are in 10 shown.

10 Figure 12 is a simplified schematic showing the combination of the SVM model with the RR estimation for a more robust extraction of a respiratory rate.

Due to the large proportion of movements (more than half), not every minute has an associated RR result. It is observed that during the intervals when the participant was mostly resting, that is, during normal breathing within the first 20 minutes, deep breaths from the 38th minute and shallow breaths from the 53rd minute onwards RR with greater accuracy through both peak counting 33 as well as HT 34 was extracted. Then the SVM model was replaced by the first model of a neural network ψ ^(mlp) (·). As in 8th shown, the MLP model had an acceptance-rejection pattern very similar to that of SVM which was also confirmed by similar quantitative evaluation results for error, precision and hit rate. As a result, the MLP integration for the RR estimation gave results very similar to those in FIG 10 is shown. Captured by all RR associated with the same timestamps SVM and MLP an absolute average difference of 0.56 breaths per minute and 0.42 breaths per minute for the peak count 33 or. HT 34 . Due to their similar results, the RR estimate combined with the LMP model is not plotted.

The hybrid RNN on the other hand, showed higher detection accuracy than both the SVM and MLP models, as in 8th is shown. Then the SVM model was replaced by the RNN model, and that ultimately extracted RR is in 11th shown.

11th Figure 13 is a simplified schematic showing the combination of the hybrid RNN model and the RR estimate. In comparison with 10 Fewer segments were assigned to an RR prediction because more frames were recorded correctly as unreliable. When the participant breathed normally at rest, ie the first 20 minutes, the hybrid RNN model showed almost identical RR values as the SVM model. Interestingly, if the participant was breathing very quickly around the 53rd minute, the peak count became RR = 41 breaths per minute 33 only 34 Peaks counted. This capture rate is because a constraint was imposed that only if the signal fell on either side by at least the standard deviation of that one minute segment, it was considered a spike. This restriction is appropriate when a person is breathing normally, with full cycles of breathing having similar peaks. However, there would be several relatively shallow breaths of magnitude during the rapid breathing period A. is very small. In this way, these associated peaks have been neglected. The top count 33 did not introduce significant error when using the SVM model as frames of these shallower breaths were recorded as unreliable (false negative result). In contrast to the method of peak counting 33 , showed the HT 34 greater robustness to dynamic levels of breathing.

wobei RR^(gth) die Grundwahrheit ist und | · | der Modul-Operator ist. Die durchschnittlichen Ergebnisse wurden in Tabelle III berichtet.For the quantitative evaluation of the RR estimation accuracy, the estimated RR compared to the ground truth RR counted by a trained observer. To reduce the error of human observation, the observer counts breaths during the two minute length instead of one as is done in conventional hospital measurements. A total of 28 counts lasting 56 minutes were obtained. The following evaluation metric was applied: $$\text{e}=\frac{\left|\text{RP}-{\text{RR}}^{\left(\text{gth}\right)}\right|}{\left|{\text{RR}}^{\left(\text{gth}\right)}\right|},$$

where RR ^{(gth) is} the fundamental truth and | · | is the module operator. The average results were reported in Table III.

Table III: Average RR estimation error e from 56 minutes. direct SVM MLP Hybrid RNN Peak counting 18.26% 6.54% 5.10% 5.53% HT based 10.43% 5.48% 4.40% 3.92%

Where the procedure of RR estimation of the peak count 33 or the HT 34 applied directly to the collected waveform, ie "direct" in Table III, a high error was obtained, especially for the peak count 33 (almost 20%). Even if the HT-based method 34 showed some improvement over the peak count, there was still an error of about 10%. These high errors are in with the large variations from one frame to another 9 consistent. When the revealed When data reliability capture was added to the RR estimation process, the error was significantly reduced, especially when the hybrid RNN classification model was combined with the HT-based 34 RR estimation. It should be emphasized that nonetheless, the basic truth is used as the gold standard. However, it does not take into account the likelihood of human error in observation. For example, only complete cycles of breaths were counted, while only partial breaths were neglected, making an under-calculation more actual RR was introduced. All three models tested showed a significant improvement in RR estimation accuracy, demonstrating the effectiveness of the disclosed strategy.

In order to justify the practicability of the described RR estimation method, the computational complexity analysis for processing data for one minute (60 frames, 600 raw waveform data points) was performed, which is listed as follows. The number of operations in the feature extraction from the raw data is about 60 × Q (2L + 1) 10. The ten-fold SVM with the RBF kernel had approximately 10 × 60 × n _sv Q operations, where n _{sv is} the number of support vectors for each group (fold), which in the trained SVM model averages 7876 when L = 3 and Q = 15. The computationally intensive operations in the proposed MLP lie mainly between the three fully connected hidden layers, each spanning the size of 100, the number of which is approximately 60 × 2 × 100 ² . Most of the RNN operations exist in the two stacked and one time-shared LSTM layers, the number of which is approximately 60 x 80,000. The top count 33 has a very low computational complexity of O (600). the HT 34 has a complexity similar to the Fast Fourier Transform (FFT) plus its inverse process, which is 2O (600log ₂ 600). The total number of operations in the fragmentation line matching in the HT-based method is about 6000 when all frames are detected as reliable. As a result of this, the preprocessing, the peak count based RR estimation, and the HT based RR estimation all have very little computational complexity which is negligible compared to the three data reliability collection models. In particular, has SVM the highest processing load, about an order of magnitude higher than the model of the hybrid RNN , while the RNN model requires roughly three times more computing power than the MLP model.

A user-friendly web-based UI has also been provided to reflect best clinical practice. The front-end component is implemented in JavaScript that supports various end devices such as laptops and cell phones. Authorized users, such as doctors, clinicians and nurses, can log into the interactive UI to view and manage device and patient reports.

The present disclosure provides a loT-based bed sensor system for real-time health monitoring. There is a method for recording the respiratory rate from measurements that were collected by inconspicuous sensor devices. The method and the system have good scalability and expandability. Bed sensor devices, which are used to collect physiological signals, are non-contact and offer low operational anomalies for participants, especially for long-term disease monitoring.

The present disclosure provides machine learning models that enable simultaneous and robust RR estimation. The disclosed models are combined with a preprocessing technique for estimating the reliability and selecting the frames from raw respiratory data to contribute to the robust RR estimation. The present disclosure also gives several evaluations to show the effect of the preprocessing methods, and the respiratory rate has been tested and evaluated against the gold standard of human observation. The results show that the disclosed solution can use the data collected from the inconspicuous devices and provide an accurate estimate of a respiratory rate ( RR ) for a person lying on the bed, especially in cases where the person does not move significantly on the bed. The work has had a significant impact on providing continuous and accurate RR readings in remote disease and care monitoring and in-hospital ward monitoring. Current practice is mainly based on manual observations which are resource intensive and also subject to human error in observation and recording. In the present work the main focus was on the technical problems and the development of an end-to-end solution to transform the raw data into reliable RR information in a continuous manner.

In accordance with aspects of the present disclosure, the Internet of Things (IoT) is applied to clinical and healthcare monitoring applications, aided by the growing number of sensor devices that measure various physiological and biological characteristics be able. The present disclosure provides an IoT-based intelligent bed sensor system that collects and analyzes respiratory-related signals to provide non-intrusive monitoring in the home, hospitals, and care facilities. A contactless device is used which has four load sensors mounted under the bed and a data processing unit (data logger). Various machine learning techniques are applied to the data streamed from the data logger to determine the respiratory rate ( RR ) capture. Support vector machines (support vector machines, SVM ) and also pattern recognition methods based on neural networks (NN), which are combined either with peak detection or Hilbert transformation for robust RR calculation. Experimental results show that the methods are the RR could effectively extract using the data collected by the non-contact bed sensors. The methods are robust to outliers and noise caused by body movements. The monitoring system also provides a flexible and scalable manner for continuous and remote monitoring of sleep, exercise, and weight using the embedded sensors.

List of reference symbols

1

IoT based RR monitoring system / system

2

bed-mounted sensor / bed sensor / bed sensor unit (see also 9: local devices)

3

Data logger (see also 9: local devices)

4th

server

5

End User

6th

VPN (Virtual Private Network)

7th

WIRELESS INTERNET ACCESS

8th

RabbitMQ

9

Bed sensor device, local devices (see also 2)

10

wireless transceiver

11th

distributed devices

12th

central data processing unit / central data processing point

13th

Gateway

14th

Back-front ends (see also 17)

15th

ML component

16

MongoDB

17th

Back-end and front-end components (see also 14)

19th

Rest position in the lying position on the back

20th

Speak while lying on your back

21

Random movements

22nd

Panting

23

Dropout layer

24

Dense

25th

block

26th

Forward tensor

27

Backward tensor

28

MLP error

29

MLP loss

30th

Hybrid RNN failure

31

Hybrid RNN Loss

32

Basic truth

33

Peak counting

34

HT based

35

dashed line

A.

Magnitude

E.

Epochs

BiLSTM

bidirectional LSTM (Long Short-Term Memory)

C.

convergence

DNN

deep neural network

HT

Hilbert transformation

IoT

Internet of Things

ReLU

rectified linear units

RNN

recurrent neural network

RR

Respiratory rate

ML

Machine learning

MLP

Multi-layer perceptron (multilayer perceptron)

LR

Logistic regression

LSTM

Long short-term memory

PCA

Principal component analysis

SVM

Support vector machine

T

time

UI

User interface

Claims (17)

A method for determining a patient's respiratory rate (RR), the method comprising: measuring a patient's weight by retrieving weight-related information from a plurality of bed-mounted load sensors (2), transmitting the weight-related information to a server (4). Procedure according to Claim 1 , comprising a step of data cleansing, wherein the step of data cleansing comprises: rearranging data frames by comparing time stamps, and / or acquiring redundant data by comparing neighboring frames, and / or removing the redundant data, and / or marking up missing time frames by comparing Difference between timestamps and a total number of frames collected. Procedure according to Claim 2 comprising a step of nonlinear transformation using a sigmoid function to amplify signals caused by small magnitude (A) breathing while suppressing a body movement signal obscuring the breathing information. Procedure according to Claim 3 , comprising the step of segmentation with overlapping, movable rectangular windows that extend over several, for example 1, 2, 3, 4, 5, 6 or in particular 7, frames with the step of 1 frame, further comprising a principal component analysis for dimension reduction . A method according to any one of the preceding claims, comprising a data reliability analysis comprising: applying a binary acceptance rejection classifier to identify the reliability of each frame. Procedure according to Claim 5 , further comprising three different types of classifiers with off-line supervised training, the basic truth (32) being manually marked. Method according to one of the preceding claims, further comprising using a support vector machine (SVM). Method according to one of the preceding claims, further comprising using a multilayer perceptron (multilayer perceptron, MLP). Method according to one of the preceding claims, comprising using a hybrid recurrent neural network (Hybrid RNN). Method according to one of the preceding claims, comprising combining data frames which have been recognized as reliable into several fragments of different lengths. Procedure according to Claim 10 comprising extracting a respiratory rate (RR) using contributions from the fragments while ignoring unreliable and missing frames. Procedure according to Claim 11 comprising applying a peak detection to each fragment, tracking a weighted combination based on the length between the first and last detected peaks (if any) in each fragment and the fragment length to detect respiratory rate (RR). Procedure according to Claim 11 , comprising applying a Hilbert transform to each time domain fragment to obtain an analysis signal from which the phase information is calculated and unfolded, further comprising slope fitting for all fragments, with one (1) shared slope parameter and another distortion Parameters for each segment, with the respiratory rate (RR) being mapped from the slope parameter. Procedure according to Claim 12 or 13th , the number of frames in each block, the variation in the detected peak-to-location distance and the fitting errors being used as indicators of the level of confidence in the respiratory rate (RR). A system for determining a patient's respiratory rate (RR), the system comprising: a plurality of bed-mounted load sensors (weight sensors), which are essentially arranged at the edge of a bed, electronic circuits for retrieving and processing corresponding weight data from the plurality of bed-mounted sensors, a data logger for temporarily storing the weight data, a transmitter for transmitting the weight data to a data processing unit , wherein the data processing unit is configured to classify a slice (slice) of respiratory data in the received data, to detect a respiratory rate (RR) in the slice and to use the detected respiratory rate (RR) as a template for the detection of further respiratory rate information. System according to Claim 15 , wherein the central data processing unit is further configured to use SVM, Hybrid RNN and / or MLP for classification. System according to Claim 16 wherein the central data processing unit is further configured to apply a Hilbert transform and a fragmented line fitting.

Images