DE102018117484A1 - A method and apparatus for detecting atrial fibrillation based on a high sensitivity photomicrograph using a portable device - Google Patents

Abstract

An apparatus includes a sensor module, a data processing module, a quality assessment module, and an event prediction module. The sensor module provides biosignal data samples and motion data samples. The data processing module processes the biosignal data samples to remove a baseline and processes the motion data samples to perform a motion significance measurement. The quality assessment module generates a signal quality indicator based on the processed biosignal data sample segments and the corresponding motion significance measurement using a first depth learning model. The event prediction module generates an event prediction result based on the processed biosignal data sample segments associated with a desired signal quality indicator using a second depth learning model.

Description

Reference to related applications

This patent application claims priority of provisional U.S. Patent Application No. 62 / 581,569 entitled "Method and apparatus for detecting atrial fibrillation based on a highly accurate photoplethysmogram using a portable device" filed on Nov. 3, 2017, which is incorporated herein by reference for all purposes.

Technical area

The present disclosure relates to a medical monitoring device and method, and more particularly, to a system and method for detecting atrial fibrillation using a portable device.

Technical background

Atrial fibrillation (AFib) is one of the most prevalent cardiac arrhythmias with a prevalence of about 2% in the general population of the developed world. The presence of AFib could potentially lead to serious health risks. Traditionally, AFib is detected by an electrocardiogram (ECG). Although the ECG based approach achieves high accuracy in the detection of AFib, detecting an ECG signal requires sophisticated devices with a variety of installed electrodes and the process requires active human involvement. Recently, photoplethysmography (PPG) has become a promising alternative to ECG in AFib acquisition. The advantage of the PPG-based solution is that PPG signals can be easily detected and monitored by consumer-level portable devices that do not require active participation by the subscribers. This advantage, along with the affordable handheld devices and smartphones, can enable daily and user-friendly AFib capture and monitoring for the general public, resulting in a huge beneficial impact for the AFib-affected population.

The processing of portable device-detected PPG signals is demanding because of the various types of interference affecting signal quality. The signal quality can be drastically influenced by the quality of the sensors and movement of the user. The error noise from the sensors and user movements may obscure all signals useful for detecting AFib. Previous studies on PPG signals detected by portable devices attempted to solve this problem using a hand-made signal quality index determined by PPG data or acceleration data. In reality, however, these manually generated properties are easily susceptible to high bias and poor generalizability. The unfiltered low-quality signal degrades the performance of AFib detection in real use.

Summary

The present specification discloses, as substantially described and / or illustrated below, a method of detecting atrial fibrillation, for example in conjunction with at least one of the figures, as more fully described in the claims.

These and other advantages, aspects and novel features of the present description, as well as the details of the illustrated embodiments, may be more fully understood from the following description and the figures.

According to one embodiment, an apparatus includes a sensor module having a first sensor adapted to measure a biological signal and to measure a second sensor suitably by a motion signal, the sensor module producing biosignal data samples and motion data samples as output; a data processing module adapted to process process segments of the biosignal data samples, each segment containing biosignal data samples collected over a predetermined period of time. The data processing module is operable to subtract a signal baseline from each segment of the biosignal data samples and to produce processed biosignal data sample segments, and is also capable of generating motion significantness (motion significance measurement) for each segment of the biosignal data samples using the motion data samples collected along with the biosignal data samples; The motion importance measurement may be an example of the degree of movement during detection of the respective segment of biosignal data samples. The apparatus also includes a quality assessment module adapted to generate a signal quality indicator based on the processed biosignal data sample segments and the corresponding motion importance measurement using a first deep learning model; and an event prediction module adapted to provide an event prediction result based on the processed biosignal data sample segments having a desired biosignal data sample segment Signal quality indicator are linked using a second depth learning model to produce.

list of figures

• 1 zeigt eine Vorrichtung gemäß Ausfiihrungsformen der vorliegenden Erfindung.
• 2 zeigt ein Blockdiagramm einer elektronischen Vorrichtung gemäß Ausführungsformen der vorliegenden Erfindung.
• 3 zeigt ein AFib-Erfassungssystem gemäß Ausführungsformen der vorliegenden Erfindung.
• 4 zeigt ein Flussdiagramm, das ein Verfahren zur AFib-Erfassung und einer tragbaren Vorrichtung gemäß Ausführungsformen der vorliegenden Erfindung zeigt.
• 5 zeigt Beispiele verschiedener PPG-Datenprobensegmente, die mit verschiedenen Qualitätswerten als Qualitätslabels gekennzeichnet sind.
• 6 zeigt beispielhafte Kurvenformen der Bewegungswichtigkeitsmessung eines 30-Sekundensegments eines PPG-Signals
• 7 zeigt die Architektur eines Qualitätsprüfungsnetzwerks in einigen Ausführungsformen
• 8 zeigt die Architektur eines AFib-Vorhersagenetzwerks unter Verwendung des CRNN-Modells gemäß einiger Ausführungsformen.

Various embodiments of the present description are described in detail in the following description and the accompanying drawings.

• 1 shows a device according to embodiments of the present invention.
• 2 FIG. 12 is a block diagram of an electronic device according to embodiments of the present invention. FIG.
• 3 shows an AFib detection system according to embodiments of the present invention.
• 4 FIG. 12 is a flowchart showing a method of AFib detection and a portable device according to embodiments of the present invention. FIG.
• 5 shows examples of different PPG data sample segments labeled with different quality values as quality labels.
• 6 FIG. 12 shows exemplary waveforms of motion importance measurement of a 30 second segment of a PPG signal
• 7 shows the architecture of a quality inspection network in some embodiments
• 8th FIG. 12 shows the architecture of an AFib predictive network using the CRNN model according to some embodiments.

Detailed description

The present disclosure may be implemented in many ways, including but not limited to, a process, apparatus, system, material composition, computer program product on a computer readable storage medium, and / or a processor, such as a hardware processor or processor device execute commands stored and / or provided on a memory connected to the processor. Throughout this description, these implementations or any other form that the present description may take is referred to as a technique. In general, the order of the steps of the disclosed processes may be changed within the scope of the present description. Described may be a component, such as a processor or memory described as configured to perform a task, implemented as a generic component that is configured to perform the task at a particular time or as a specific component that has been manufactured Perform task. As used herein, the term "processor" refers to one or more devices, circuits, and / or processor cores suitable for processing data, such as computer program instructions.

A detailed description of one or more embodiments of the present invention is provided below together with the accompanying drawings which illustrate the principles of the present specification. The present disclosure is described in connection with such embodiments, but the present description is not limited to any embodiment. The scope of the present description is limited only by the claims, and the present disclosure includes a variety of alternatives, modifications, and equivalents. Many specific details are set forth in the present description to provide a thorough understanding of the present description. These details are exemplary and the present description may be practiced according to the claims without some or all of these specific details. For the sake of clarity, the technical material known in the technical fields with respect to the present specification has not been described in detail, so that the present description will not be unnecessarily complicated.

In some embodiments of the present invention, a user-portable device implements a cardiac flicker (AFib) detection system to provide highly accurate AFib detection using a channel of a biological sensor signal and a motion signal, with only the computer power provided by the portable device. According to some embodiments, the portable device includes a biophysiological sensor and an inertial measurement sensor for collecting the biological signal and the motion signal of an object carrying the user-portable device. The user-portable device also implements a neural network using deep learning models to detect AFib using the biological signal and the motion signal. According to some embodiments, the biophysiological sensor is a photoplethysmography (PPG) sensor and the initial measurement sensor is an accelerometer. In this way, the portable user-friendly device provides a portable non-destructive device for monitoring and detecting AFib at a low cost, allowing access to the general population. In other embodiments, the user-portable device may be adapted as a medical observation device for detecting other biological sensor signals and using the detected biological sensor signals with the movement data of the subject to detect a medical event of interest. The configuration and application to a user-portable device of the present disclosure is not limited to ventricular fibrillation detection and may have other applications in health observation.

The user-portable AFib recognition device according to the present invention achieves many advantages over ordinary systems and methods. First, the AFib detection system according to the present invention uses a single channel of biological signal and a single channel for the motion signal and a lightweight AFib detection method which directly applies neural networks to the raw signals of the sensors with a minimum of pretreatment or conversion. Accordingly, the AFib recognition system can be effectively implemented in a portable device while still enabling very accurate AFib recognition. Second, the AFib recognition system evaluates the biological signal with the motion signal using a deep learning model to assess the quality of the biological signal without manually using quality measurements. In this way, only good quality biological signals are used for AFib detection, thus achieving solid AFib prediction results. According to one embodiment, the AFib recognition system includes a quality assessment network implemented by training an auxiliary convolutional neural network using a raw biological sensor signal as input to accurately evaluate the quality of the signals collected by the biophysiological sensor without manually performing quality measurements. The quality assessment network operates as a gatekeeper and identifies raw biological sensor signals with good signal quality, which are then suitable for AFib detection. The AFib detection system also includes an AFib prediction network that is another trained CNN and that predicts the probability of occurrence of AFib in the signals provided by the quality assessment network. The AFib recognition system of the present invention achieves high accuracy in AFib recognition comparable to the prior art ECG-based approaches, but has a simple structure that can be implemented even in small mobile devices such as a portable wrist device.

1 shows an electronic device according to embodiments of the present invention. 2 FIG. 12 is a block diagram of an electronic device according to embodiments of the present invention. FIG. Referring to the 1 and 2 has an electronic device ( 100 ), which may be a user-portable device, an indicator ( 160 ), a processor ( 130 ), a sensor module ( 150 ), a battery, a band ( 140 ), and a closure ( 142 ). The ribbon ( 140 ) can be placed around a wrist and the user-portable device ( 100 ) may be attached to the wrist using the closure ( 142 ) being held. The sensor module ( 150 ) contains one or more sensors ( 152 . 156 ) and a local processor ( 154 ). The local processor ( 154 ) implements a control function for the sensor module and may also perform processing or preprocessing of the detected signals. The processor ( 130 ) implements control functions for the user-portable device and may also continue to perform signal-processing functions on the detected signals. The local processor location ( 154 ) or the processor ( 130 ) may also be referred to as diagnostic processors.

Although the user-portable device ( 100 ) can be worn on a wrist, the present description is not limited to this embodiment. The user-portable device ( 100 ) can also be designed around other parts of the body, such as an arm (around the forearm, elbow, or upper arm), on one leg, on the chest, on the head like a headband, on the neck like a collar, and worn on the ear. The user-portable device ( 100 ) may also communicate with other electronic devices such as a smartphone, a laptop, or various medical devices in the hospital or doctor's office.

The ad ( 160 ) may output sensed physiological signals of the user's body for observation to the user and / or others. The observed physiological signals are also referred to as biosignals or biometric data. The observed biosignals may be, for example, heart (pulse) rate, pulse morphology (shape), pulse interval (inter-stroke intervals), respiratory (breath) rate, or blood pressure. The ad ( 160 ) may also command the user or others to use the user-portable device ( 100 ) or for use in other measuring devices, or as a status or diagnostic result, for example.

The processor ( 130 ) receives the observed and detected signals of the sensor in the sensor module ( 150 ). For example, the sensors detect ( 152 . 156 ) Signals from the user's wrist when the user-portable device ( 100 ) is worn by a user. In embodiments of the present invention contains the sensor module ( 150 ) a first sensor ( 152 ) as a biophysiological sensor and a second sensor ( 156 ) as initial measuring sensor. In one embodiment, the biophysiological sensor is a photoplethysmography (PPG) sensor and the inertial measurement sensor is an accelerometer. The sensor module ( 150 ) contains the processor ( 154 ) for controlling the sensors ( 152 . 156 ) and for processing the signals generated by the sensors ( 152 . 156 ) were recorded. For example, the processor ( 154 ) from the sensors ( 152 . 156 ) and then reconstruct the separated signal. Various embodiments of the description include the processor ( 130 ) on the functions of the processor ( 154 ). Various embodiments of the disclosure may also include a different number of sensors.

According to some embodiments, the sensor ( 152 ) a PPG sensor used to continuously or periodically acquire cardiac physiological information, such as a user's heart rate or cardiac output. The sensor ( 156 ) may be an acceleration sensor for continuously or periodically detecting movement information of a user. The sensor module ( 150 ) may also include other sensors such as a thermometer to detect the temperature of the user.

The user-portable device ( 100 ) implements the AFib acquisition system according to the present invention in the processor ( 130 ). The AFib acquisition system includes a quality assessment network for evaluating the quality of the biophysiological signal generated by the biophysiological sensor (FIG. 152 ) was measured using the motion signal generated by the Innertialmesssensor ( 154 ), and also includes an AFib prediction network for evaluating the biophysiological signals that are acquired to have good quality and to predict the probability of AFib occurrence in the observed signal. The detailed structure of the AFib detection system will be described in more detail below.

Referring again to 2 contains a user-portable device ( 100 ) a sensor module ( 150 ), a processor ( 130 ), an ad ( 160 ), and a battery ( 170 ) to power the other components. The processor ( 130 ) controls the output to the display ( 160 ). The ad ( 160 Also includes input devices (not shown) such as buttons, scales, a touch-sensitive screen, and a microphone.

According to embodiments of the present invention, the sensor module ( 150 ) at least one biophysiological sensor ( 152 ) to measure a biological signal of the user. In the present embodiment, the biophysiological sensor ( 152 ) a PPG sensor. Sensor module ( 150 ) further comprises at least one inertial measuring sensor ( 156 ) to measure a motion signal of the user. In the present embodiment, the inertial measuring sensor ( 156 ) An accelerometer such as a triaxial accelerometer. The sensor module ( 150 ) can be used with at least one local processor ( 154 ) be equipped around the sensors ( 152 . 156 ) and, in addition, the biosignals and motion signals of the sensors ( 152 . 156 ) to process accordingly. In some embodiments, the signal processing in the local processor ( 154 ) and / or the processor ( 130 ).

Alternatively, the local processor ( 154 ) perform some signal processing such as reliable signal preprocessing, and the processor ( 130 ) implements other signal processing algorithms for biometric detection and / or for other functions. In embodiments of the present invention, the use of a special processor to implement the biometric signal processing algorithms is not critical to practicing the present invention.

According to embodiments of the present invention, the processor ( 130 ) is configured around the capture process, the scan process, the signal processing operation, and device communication events and other device-specific functions in the user-portable device ( 100 ) to control. In the present embodiment, the processor includes ( 130 ) a CTU ( 132 ), a memory ( 134 ), and an input / output (I / O) interface ( 182) a communication interface ( 184 ), and an AFib acquisition module ( 190 ). While the processor ( 130 ) is described as containing these various devices, other embodiments may use other architectures in which the different functionalities are grouped differently. For example, the grouping can be implemented in different integrated circuit chips. The grouping can also be achieved by the combination of various devices, such as the I / O interface ( 182 ) and the communication interface ( 184 ) consist. The memory ( 134 ) contains at least one of a nonvolatile memory ( 137 ) and a volatile memory ( 138 ). The memory ( 134 ) can contain data or commands of an operating system (OS) ( 153 ) or applications ( 136 ) to save. The processor ( 120 ) contains the AFib acquisition module ( 190 ) around the AFib Detecting the detected biosignal such as the PPG signal using the motion signal such as the motion signal from the accelerometer. In embodiments of the present invention, the AFib acquisition module ( 190 ) a data processing module ( 192 ), a quality assessment module ( 194 ) and an AFib prediction module ( 196 ). The data processing module ( 192 ) is configured to perform the signal preprocessing of the detected biosignal. For example, the data processing module ( 192 ) perform the baseline removal or the removal of the DC signal level of the detected biosignal. The quality assessment module ( 194 ) implements a quality rating network that acts as a gatekeeper and identifies segments of good quality PPG raw signal and is therefore suitable for AFib acquisition. The AFib acquisition module (196) implements an AFib prediction network that predicts the probability of AFib occurrence in the good quality signals provided by the quality assessment network.

3 shows an AFib detection system according to embodiments of the present invention. In some embodiments, the AFib detection system is off 3 as an AFib acquisition module ( 190 ) in the processor ( 130 ) of the user-portable device ( 100 ) implemented. Referring to 3 is the AFib acquisition system ( 200 ) in communication with a sensor module ( 150 ) to receive the detected biological signal and motion signal. In the present example, the sensor module ( 150 ) a PPG sensor ( 152 ), which provides a PPG signal as a biological signal and an accelerometer ( 156 ), which provides an acceleration signal as a motion signal. The sensor module ( 150 ) contains a local processor ( 154 ) to provide reliable signal preprocessing and control functions. It is clear that the biological signal and the motion signal are both time-sequential signals and are measured continuously at electrical voltage levels at a given sampling rate in the usual way.

Accordingly, in operation, a biological signal measured by a biophysical sensor is provided as a time series of biosignal data samples. Likewise, a motion signal measured by an intertial measurement sensor is provided as a motion data sample. In the present example, the PPG sensor ( 152 ) and the accelerometer ( 156 ) at a sampling rate of 100 hertz, which means 100 samples / second.

In the present example, the PPG sensor ( 152 ) Raw PPG data (node 152 ) as output while the accelerometer ( 156 ) Acceleration raw data samples (node 157 ) as output. In the present specification, the term "raw" signal refers to a detected signal which has not been processed or has been minimally processed.

In the present specification, the observed biological signal is referred to as the PPG signal, however, it will be understood that other types of biological signals may be used in the AFib detection system of the present invention. The use of the PPG sensor ( 152 ) is exemplary only and is not intended to be limited thereto. In other embodiments, the biological signal may be another cardiac-related physiological signal. In addition, in the following description, the motion signal through an accelerometer ( 156 However, it is clear that other types of motion sensor or inertial measurement sensor can be used to obtain the motion signal. The use of the accelerometer ( 156 ) is merely exemplary and is not intended to be limited thereto.

The AFib acquisition system ( 200 ) receives the PPG raw data samples and the accelerometer raw samples. A salient feature of the AFib acquisition system ( 200 ) of the present invention is that the AFib detection system ( 200 ) requires only a single channel of the PPG data samples and a single channel of accelerometer samples to provide very accurate AFib prediction results. Accordingly, the AFib acquisition system ( 200 ) with a low computational structure, making it suitable for implementation in a portable device. In the AFib acquisition system ( 200 ) are the PPG raw data samples for a baseline removal module ( 202 ) to remove a baseline signal level or a DC offset from the signal. For example, the baseline signal level is removed so that the PPG data samples have voltage values that are around 0 volts. In some embodiments, the PPG data samples are processed as segments of the PPG data samples, each segment containing PPG data samples collected over a given period of, for example, 30 seconds. At a sampling rate of 100 hertz contains each segment 3000 PPG data samples. The baseline removal module ( 202 ) works with PPG data sample segments and produces as output value processed PPG data sample segments. Meanwhile, raw acceleration data samples for evaluation for a motion significant modulus ( 204 ) provided. For each segment of the PPG data samples, there is a corresponding segment of motion data samples collected concurrently with the PPG data samples become. The motion significance module ( 204 ) generates a motion significance measurement for each segment of PPG data samples using the corresponding motion data samples. In particular, the motion significance measurement is indicative of a degree of movement during the detection of the corresponding segment of the PPG data samples. In one embodiment, the motion significance measurement includes a first motion indicator indicative of moderate motion and a second motion indicator indicative of strong motion. The motion significance module ( 204 ) evaluates the motion data samples and classifies the motion data samples corresponding to a corresponding segment to PPG data samples using the first motion indicator or the second motion indicator. In one embodiment, the raw motion data samples are processed in segments in the same manner as the raw PPG samples. In one embodiment, the motion significant module ( 204 ) the motion significance measurement (node 208 ) for each segment of the motion data samples based on the magnitude in three axes of the motion signal measured by the accelerometer.

The AFib acquisition system ( 200 ) contains the quality assessment network ( 110 ) receiving the processed PPG data sample segments and the corresponding motion significance measurement. The quality assessment network ( 210 ) evaluates the processed biosignal data sample segments with the motion significance measurement using a first depth learning model that was previously trained based on a set of quality levels and one or more sets of quality training data. In operation, the quality assessment network ( 210 ) identifies as output a signal quality indicator in which the signal quality indicator identifies each processed biosignal data sample segment as having "good signal quality" or "poor signal quality". In practice, the quality processing network ( 210 ) PPG data sample segments with poor signal quality (node 214 ) and allows only PPG data sample segments ( 212 ) with good signal quality to move to the next level for AFib prediction. Since only good quality PPG data sample segments are used in AFib prediction, the AFib acquisition system ( 200 ) able to make a stable and reliable AFib prediction.

A salient feature of the quality assessment network ( 210 ) is the use of motion significance measurement (node 208 ) in assessing the signal quality of the PPG data samples. The motion data samples are not only used to set a good and bad signal quality threshold. Much more, the motion data is used in conjunction with the morphology of the PPG data samples and applied to the first depth learning model to classify the quality of the PPG data samples. For example, in some cases, even if the motion is fierce, the signal quality can still be good. Alternatively, in some cases, even if the movement is moderate, the signal quality may be poor. The first deep learning model classifies the quality of the PPG data samples in terms of motion data to provide meaningful quality assessment results. The structure of the quality assessment network ( 210 ) will be described in detail later.

The AFib acquisition system ( 200 ) contains the AFib forecasting network ( 220 ) that contains the processed PPG data sample segments that have been evaluated by the quality assessment network ( 210 ). That is, the AFib predictive network ( 220 ) receives only processed PPG data sample segments that have been evaluated with good signal quality. The AFib forecasting network ( 220 ) evaluates the processed biosignal data sample segments using a second depth learning model that was previously trained based on AFib annotations and one or more sets of AFib acquisition training data. The AFib forecasting network ( 220 ) is configured by a prediction result (node 222 ), which indicates the likelihood that arterial flicker will be present in a given segment of biosignal data samples. In response to AFib being present in a particular segment of biosignal data samples, the AFib acquisition system ( 200 ) a message to the user. For example, the message may be sent via an application on a mobile device and / or the portable device.

In an alternative embodiment, the AFib predictive network ( 220 ), the motion significance measurement and the PPG data sample segment in good quality, together with the corresponding motion significance measurement, is then assessed using the second depth learning model.

In one embodiment, the first depth learning model is in the quality assessment network ( 210 ) as a multi-modal neural network having three convolutional layers with max-pooling, batch normalization and rectified linear unit as activation function followed by a fully-connected layer and an output layer, wherein the output of the convolutional layers is smoothed and processed with the motion-significance measurement Biosignal data sample segment is linked and transported into the fully-bonded layer.

In another embodiment, the second depth learning model is in the AFib predictive network ( 229 ) is implemented as a hybrid model of a convolutional neural network (CNN) and a recurrent neural network (RNN), wherein each segment of the processed biosignal data samples is divided into a plurality of sub-segments in which each sub-segment is transported in timed order into locked recurrent units.

4 FIG. 12 is a flowchart illustrating a method of AFib detection in a user-portable device according to embodiments of the present invention. FIG. In some embodiments, the method ( 300 ) in a processor of a portable device, such as the processor ( 130 ) of the user-portable device in the 1 and 2 be implemented. Referring to 4 begins the procedure ( 300 ) receiving a signal channel of biosignal data samples of a first sensor used in the user-portable device ( 302 ) is implemented. For example, the biosignal data samples may be PPG data samples. The procedure ( 300 ) also receives a single channel of motion data samples from a second sensor included in the user-portable device (FIG. 302 ) is implemented. For example, the motion data samples may be accelerometer data samples. In the present embodiment, the method receives ( 300 ) Raw data samples, ie data samples that are not processed or only minimally processed.

at 304 performs the procedure ( 300 ) performs baseline signal level removal on segments of the biosignal data samples to provide processed biosignal data sample segments. at 306 classifies the procedure ( 300 ) a degree of motion associated with each segment of the biosignal data samples using the motion data samples acquired simultaneously with the biosignal data samples. In one embodiment, the degree of movement is classified into a first movement indicator, which indicates a moderate movement, and a second movement indicator, which indicates a violent movement. In one embodiment, the method classifies ( 300 ) the degree of movement associated with each segment of the biosignal data samples has been detected based on an average of the magnitudes in three axes of the motion signal of the accelerometer.

The procedure ( 300 ) goes with it 308 wherein the processed biosignal data sample segments and corresponding motion indicators for the respective segments are provided to a quality assessment network that implements a first depth learning model that was previously trained based on a set of quality labels and one or more sets of quality training data. at 310 generates the procedure 300 a signal quality indicator that identifies each processed biosignal data sample segment with good signal quality or poor signal quality. Biosignal data sample segments of poor quality are sorted out ( 312 ).

The procedure ( 300 ) goes to step 314 in which processed biosignal data sample segments with associated signal quality indicator indicative of good signal quality are provided to an AFib predictive network implementing a second depth learning model previously trained based on AFib annotations and one or more AFib acquisition training data sets. At step 316 generates the procedure ( 300 ) a prediction result indicating a probability that arterial flicker is present in a given segment of biosignal data samples. In some embodiments, the AFib detection method ( 300 ) in response to detecting that an AFib is present an indication to the user. For example, the indication may be sent via an application on a mobile device and / or the portable device.

In a further embodiment, the method ( 300 ) the processed biosignal data sample segment with associated signal quality indicator indicating good signal quality and the associated motion indicators for the corresponding segments are available to the second depth learning model.

In some embodiments of the present invention, the quality assessment network or module applies the first depth learning model to evaluate the PPG data samples and to determine a good or poor signal quality level. The first deep learning model was previously trained based on a set of quality markings. In one example, a set of PPG data is collected, and the signal quality of the collected PPG data is evaluated by a human expert, and a quality index from 1 (lowest quality) to 10 (highest quality) can be assigned to each segment as a quality level. For example, the quality of the signal may be based on the morphology, periodicity, presence of a dicrotic incision, and the consistency of the beats. 5 shows examples of different PPG data sample segments labeled with different quality values as quality levels. Referring to 5 signal segments from the left column are marked as non-AFib and those from the right column are labeled AFib. If the quality of Signal is low, it is impossible to distinguish AFib signals from non-AFib signals.

The term "deep learning model" as used herein refers to the classification of models that require extended training times for more accurate classifications. In practice, the training of a classification model is performed on high performance computers and the trained model is then transferred to the device where conclusions are then made using the model. In some embodiments, deep learning neuronal networks, as used herein, may be referred to as a deep learning model. However, other machine learning and / or classification techniques may be used to generate a depth learning model. Briefly, embodiments of the present invention relate to AFib detection or event prediction using machine learning that is progressively improved based on expert contribution. In at least one of the various embodiments, data is provided for a depth learning model that has been trained using a plurality of classifiers (indexes, labels or annotations) and one or more sets of training data and / or test data.

The detailed construction of an AFib detection system according to the present invention will now be described with reference to an embodiment using a PPG sensor as a biophysiological sensor. The embodiments described below are merely exemplary and should not be construed as limiting.

According to an exemplary embodiment, PPG data of 19 Patients with 1443 PPG segments captured by a portable device. Each PPG segment contains 30 seconds of data at a sample rate of 100 Hertz and so 3000 Samples per segment. In addition, each PPG segment has acceleration values that are concurrent with 100 Hertz were recorded. Out of all the PPG samples, have 1101 from them AFib and 342 have different rhythms. In addition, a quality index in the range of 1 (lowest quality) to 10 (highest quality) manually noted by human experts and provided with each PPG sample. The quality of the signal can be evaluated based on factors such as periodicity, presence of a decrotic indentation and the consistency of the beats. 5 shows exemplary waveforms of the PPG signals. ECG samples are also detected and labeled with the presence AFib sequence. Such labels are used as the basic correctness for AFib.

data processing

• $Acc=\sqrt{x^2+y^2+y{z}^2}$
  
  des Beschleunigungsmesssignals. Insbesondere enthält jedes Bewegungssegment Ergebnisse in zwei Bewegungsindikatoren - gemäßigte Bewegung und starke Bewegung - die durch Schwellwertbestimmung des Teils der Standartabweichungen eines Halb-Sekundenfensters über ein 30-Sekunden-PGG-Segment berechnet werden. Die Schwellwerte können vorab bestimmt werden. Außerdem kopiert das AFib-Erkennungssystem die Signalsegmente mit Qualitätsindex größer als 6 in gute Qualitätssegmente (gekennzeichnet als 0) und den Rest als schlechte Qualitätssegmente (gekennzeichnet als 1) wodurch sich beispielsweise 716 gute Qualitätssegmente und 727 schlechte Qualitätssegmente ergeben.

In one embodiment, the AFib detection system applies eight level elementary wave decomposition and Daubechies elementary waves to each segment of the PPG, and the approximation channel is removed in the elementary wave reconstruction to remove the baseline. A set of motion significance measurements is calculated for each segment based on the corresponding magnitude of the acceleration measurement signal to detect the degree of motion during the recording of the PPG signal. For example, the motion significance measurements can be calculated as an average of the magnitudes in three axes:

• $Acc=\sqrt{x^2+y^2+y{z}^2}$
  
  of the acceleration measurement signal. In particular, each motion segment contains results in two motion indicators - moderate motion and strong motion - which are calculated by thresholding the portion of the standard deviations of a half-second window over a 30 second PGG segment. The thresholds can be determined in advance. In addition, the AFib recognition system copies the signal segments with quality index greater than 6 into good quality segments (marked as 0 ) and the rest as bad quality segments (marked as 1 ) which, for example, causes 716 good quality segments and 727 result in poor quality segments.

6 FIG. 12 shows exemplary waveforms of a motion-significance measurement on a 30-second segment of a PPG signal. FIG. The left waveform represents a raw acceleration signal. The right waveform shows 0.5 second window with high motion ( 402 ) and moderate movement ( 404 ). The division of the windows into strong and moderate motion (shown above) is used as a motion significance measurement as described above.

Quality Evaluation Network

7 shows the architecture of the quality assessment network in some embodiments. Referring to 7 For example, the quality assessment network includes a multi-modal neural network designed to accept two inputs, the 30 second preprocessed PPG signal and the aforementioned strong motion measurement, and predicts a probability of whether or not a signal is of good quality. The motion variable is introduced in the last step together with the learned properties of the PPG of the convolutional layers. The model is trained by linear good / bad quality levels described above.

This in 7 shown quality assessment network consists of 3 convolutional layers with max-pooling, batch normalization and rectified linear unit (ReLu) as the activation function, followed by a fully connected layer and an output layer. There are 8th . 8th and 4 Filters with sizes of 32 . 8th and 2 along the temporal dimension in the first, second and third layers. The pooling sizes of the 1D max pooling layers are appropriate 12 . 8th and 2 , The samples are not filled for filters. The output of the convolutional layers is smoothed and associated with the motion significance measurement for that signal and is delivered to a fully connected layer with ReLU activation, followed by the output layer with a sigmoid activation function that outputs the probability of the poor quality segment.

The model is adjusted for an L2 constraint override applied to the weighting of all convolutional filters and to avoid drop-out regulation on the fully connected layer. The parameter for the L2 constraint is 0.05 and the rate for drop-out is 50%. The total number of parameters of this model is 2.529.

AFib Forecast Network

In some embodiments of the present invention, the AFib predictive network includes a second neural network that accepts the preprocessed PPG signal predicted by the quality assessment network as a good signal. The model is trained with AFib designation on the corresponding 30 second ECG signal as the basic accuracy objectives, according to one embodiment. To effectively extract both local waveform structures and their temporal progression as properties for predicting AFib, we developed a Convolution Recurrent Hybrid Model (CRNN) model, which is a hybrid model of CNN and develops a recurrent neural network (RNN). The AFib prediction network may also be implemented using a CNN model with no recurring structures as a baseline or as types of deep neural network architectures.

In one embodiment, the CNN baseline model is a three-layer CNN, similar to the Quality Assessment Network (see 6 ). The difference lies in the number of filters in each convolution layer, this model 32 . 16 and 16 Contains filter. In particular, more filters are needed to extract complex waveform structures that are invariant in the AFib PPG segments. In addition, in the present embodiment, no motion measurement is connected as a second input. The total number of paraments of this model is 13 . 121 , In other embodiments, the motion measurement may be linked as a second input to the CNN baseline model.

8th illustrates the architecture of the AFib predictive network using the CRNN model in some embodiments 8th The CRNN model includes an RNN structure with signals from consecutive time windows as input to better model the complex time dependencies of the PPG signal, such as irregular patterns between fixed time segments. In the CRNN model, the 30-second segment of the data samples is divided into ten 3-second window segments, and each segment is entered in chronological order into CNN Property Tractor GRU.

To efficiently extract waveform structures from each time window, a property extractor containing three convolutional layers is applied to each window. The number of filters is 32 . 16 and 16 for the first, second and third layers of CNN. The size of the filters along the temporal dimensions are 32 . 16 and 8th and the size of the 1D max-pooling layer is 4 . 4 and 2 for each level accordingly. The samples are not filled up for the filters. The weights of the filters in the CNN are shared over all temporal steps, provided that the properties with respect to the waveform structures are invariant along the time dimension. Rectified Linear Units (ReLU) are used as the activation function for all convolutional layers. The extracted renderings from each window are then entered into Gated Recurrent Untis (GRU) at each temporal step (3 seconds). The size of a hidden layer in each GRU cell is 16 , As in 8th shown is a break (50%) applied to the hidden layer of recurring units.

optimization

The weighting of each model was optimized by an Adam optimizer (learning rate of 0.0005) via backpropagation, using binary cross entropy as a loss function. To address the unbalanced class sizes in the AFib acquisition tasks, a higher class weight is assigned to the under-represented classes (segments without AFib) of 0.7 and the other classes of 0.3 when the AFib acquisition models are trained. The models can be equipped with mini-batches of 64 Rehearsals are trained. The training ends at 150 Epochs or each time the validation error for ten consecutive iterations does not improve anymore to avoid overfitting.

Quality rating results

The current system can use a five-fold cross validation to evaluate the performance of the deep learning model. The present system may use an average accuracy of the test set in each convolution (ACC) and a pooled AUC in the predicted probabilities of the test set in each convolution. To demonstrate the effectiveness of the model, a simple rule-based baseline model can be used in which the threshold of motion measurements is set to separate good and bad quality signals. In one embodiment, the present system also tests two alternative models: a CNN model without adding motion measurements as input, and a CNN model using both a raw accelerometer signal and a PPG signal. The evaluations are tested as basic truth compared to human assessment. The depth learning model for quality determination as used here has a much higher accuracy and AUC than the baseline model achieves the AUC of 95.21%.

AFib detection results

In order to independently evaluate the AFib acquisition system, all signals marked with good quality (with a quality value> 6) will be used to train and validate the AFib prediction models. The resulting signal samples are dominated by AFib samples. To train more stable models, a higher weighting is applied to the under-represented classes in loss functions of the models.

Two models were tested including the above two architectures CNN and CRNN. Both models have an accuracy greater than 97% and AUC greater than 99%. The hybrid model achieved the best performance everywhere, proving that modeling the time-dependency information is helpful in predicting AFib. The result shows that a relatively small network structure with a number of parameters in the range of 10,000 can achieve very high performance.

In embodiments of the present invention, the depth learning AFib acquisition system using a PPG signal exhibited very high performance, with a minimum of feature narrowing by training a quality assessment network, the system is able to properly filter and provide bad signals thereby ensuring a solid prediction through the AFib detection algorithm. This is particularly effective when the signals come from a portable device and are subject to high quality variation. Even a small convolutional network can therefore predict AFib with great accuracy.

Aspects of this disclosure are described with reference to flowchart illustrations or block diagrams in which each block or combination of blocks may be implemented by computer program instructions. The instructions may be provided to a processor or general purpose computer, special purpose computer or other programmable computing device to form a machine or an article of manufacture, and in the case of execution by the processor, the instructions may include means for implementing the functions, acts or events contained in each block or combination at blocks specified in the diagrams are generated.

In this regard, each block in the flowchart or block diagram corresponds to a module, segment or portion of a code that contains one or more executable instructions for implementing the specified logical functions. It should be noted that in some alternative implementations the functionality associated with a block may also occur in a different order than shown in the figures. For example, if two blocks are shown in succession, they may in fact be executed substantially simultaneously, or blocks may be executed in reverse order.

One skilled in the art will recognize that aspects of this disclosure may be embodied as a device, system, method, or computer program. Corresponding aspects of this disclosure, which may generally be referred to as circuits, modules, components or systems, in hardware, in software (including firmware, resident software, microcode, etc.), or in combination of software and hardware, including computer program products on a computer-readable medium with computer-readable program code running on it.

The detailed description presented above has been provided to illustrate specific embodiments of the present invention and is not intended to be limiting. Various modifications and variations are possible within the scope of the present invention. The present invention is defined by the appended claims.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 62581569 [0001]

Claims (28)

Device comprising: a sensor module having a first sensor configured to measure a biological signal and a second sensor configured to measure a motion signal, the sensor module providing biosignal data samples and motion signal data samples as output; a data processing module configured to process segments of the biosignal data samples, each segment containing biosignal data samples acquired over a predetermined period of time; the data processing module is configured to subtract a baseline signal from each segment of the biosignal data samples and to generate processed biosignal data samples; and the data processing module is further configured to generate a motion significance measurement for each segment of the biosignal data samples using the motion data samples acquired simultaneously with the biosignal data samples, the motion significance measurement being an example of a degree of motion during detection of the corresponding segment of biosignal data samples; a quality assessment module configured to generate a signal quality indicator based on the processed biosignal data sample segments and the motion significance measurement using a first depth learning model; and an event prediction module configured to generate an event prediction result based on the processed biosignal data sample segments associated with a desired signal quality indicator using a second depth learning model. Device after Claim 1 wherein the biological signal is a cardio-related physiological signal and the motion signal includes an inertial measurement signal. Device after Claim 1 wherein the motion significance measurement includes a first motion indicator indicating a moderate motion and a second motion indicator indicating a strong motion. Device after Claim 1 wherein the device includes a user-portable device, the first sensor includes a photoplethysmography (PPG) sensor, and the second sensor includes an accelerometer. Device after Claim 4 in which the biosignal data samples contain PPG raw data samples and the motion data samples comprise raw accelerometer data samples. Device after Claim 4 wherein the data processing module is configured to generate a motion significance measurement for each segment of the motion data samples based on an average of the magnitudes of the motion signal in three axes as detected by the accelerometer. Device after Claim 1 wherein the event prediction module is configured to score the processed biosignal data sample segments with the associated motion significance measurement using the second depth learning model. Device after Claim 1 wherein the first depth learning model includes a multimode neural network having three convolutional layers with max-pooling, batch normalization and a rectified linear unit as the activation function, followed by a fully connected layer and an output layer, wherein the output of the convolutional layers is smoothed and connected to the motion significance measurement for the processed biosignal data sample segment and entered into the fully connected layer. Device after Claim 1 wherein the second depth learning model includes a hybrid model of a convolutional neural network (CNN) and a recurrent neural network (RNN), each segment of the processed biosignal data samples being divided into a plurality of sub-segments, each sub-segment being locked in temporal order recursive units is entered. Device after Claim 1 in response to the event prediction result, indicating that there is a high probability that an event exists in a given segment of the biosignal data samples, the device sends an indication to the user. Device after Claim 1 wherein the sensor module provides a single channel of biosignal data samples and a single channel of the motion data samples as output. Device after Claim 1 wherein the quality evaluation module is configured to evaluate the processed biosignal data sample segments with the motion significance measurement using the first depth learning model previously trained based on a plurality of quality levels and one or more sets of quality training data, wherein the quality evaluation module generates the signal quality indicator of each biosignal data sample segment processed having good signal quality or having weak signal quality identified. Device after Claim 12 wherein the event prediction module is configured to display the received biosignal data sample segments with the associated good quality signal of quality indicator, and to evaluate the processed biosignal data sample segments using the second depth learning model which was previously trained based on event annotations and one or more sets of event collection training data; wherein the event prediction module is configured to generate the prediction result as output, and the prediction result indicates a likelihood of whether an event in a given segment exists on biosignal data samples. Device after Claim 13 wherein the event prediction module comprises an AFib prediction module configured to generate a heartbeat (AFib) prediction result indicative of a probability that a heartbeat exists in a given segment of the biosignal data samples, wherein the AFib prediction module is configured to use the processed biosignal data sample segments using the biosignal data sample segment second depth learning model, wherein the second depth learning model was previously trained based on AFib annotations and one or more sets of AFib acquisition training data. A method of event detection in a user-portable device, comprising: Receiving biosignal data samples from a first sensor implemented in the user-portable device; Receiving motion data samples from a second sensor implemented in the user-portable device; Performing, in a processor implemented in the user-portable device, a baseline signal level removal on segments of the biosignal data samples to provide processed biosignal data sample segments; Classifying, in the processor implemented in the user-portable device, a degree of movement associated with each segment of the biosignal data samples using the motion signal data samples acquired simultaneously with the biosignal data samples, wherein the degree of movement indicates a first movement indicator indicating a moderate movement and a second movement indicator the indicating a strict movement is classified; Providing the processed biosignal data sample segments and the corresponding motion indicators for the respective segments for a first depth learning model for the processor implemented in the user portable device; Generating a signal quality indicator for each processed biosignal data sample segment in the processor implemented in the user-portable device and using the first depth learning model; Providing processed biosignal data sample segments with associated signal quality indicator indicative of good signal quality to a second depth learning model for the processor implemented in the user portable device; and Generating an event prediction result based on the processed biosignal data sample segments associated with a desired signal quality indicator for the processor implemented in the user portable device. Method according to Claim 15 wherein each segment of biosignal data samples contains biosignal data samples acquired over a predetermined period of time. Method according to Claim 15 wherein the biosignal data samples contain cardio-related physiological data samples and the kinetic data samples include inertial measurement data samples. Method according to Claim 15 wherein the first sensor includes a photoplethysmographic sensor (PPG) and the second sensor includes an accelerometer; and the biosignal data samples contain PPG raw data samples and the motion data samples contain raw accelerometer data samples. Method according to Claim 18 wherein classifying the degree of movement associated with each segment of the biosignal data samples using the motion data samples comprises: in the processor implemented in the user-portable device, classifying the degree of movement associated with each segment of the biosignal data samples based on an average of the magnitudes of that of Accelerometer detected motion signal in three axes. Method according to Claim 15 wherein providing the processed biosignal data sample segments with the associated signal quality indicator indicative of good signal quality for the second depth learning model comprises: providing the processed biosignal data sample segments with the associated signal quality indicator indicating good signal quality and corresponding motion indicators for the corresponding segments for the second depth learning model. Method according to Claim 15 further comprising: providing an indication to the user in response to the prediction result indicating a high probability that an event in present in the present segment of biosignal data samples. Method according to Claim 15 wherein: receiving the biosignal data samples from the first sensor implemented in the user-portable device comprises receiving a single channel of biosignal data samples; and receiving the motion data from the second sensor implemented in the user-portable device, including receiving a single channel of motion data samples. Method according to Claim 15 wherein: providing the processed biosignal data sample segments and corresponding motion indicators for respective segments to the first depth learning model provides the processor of the processed biosignal data sample segments implemented in the user-portable device and the corresponding motion indicators for the corresponding segments for the first depth learning model previously based based on a plurality of quality labels and one or more sets of quality training data; and generating the signal quality indicator for each processed biosignal data sample segment comprises generating for the processor implemented in the user-portable device using the first depth learning model, the quality indicator identifying each processed biosignal data sample segment as having good signal quality or poor signal quality. Method according to Claim 23 wherein providing the processed biosignal data sample segments with the associated signal quality indicator indicative of good signal quality for the second depth learning model comprises providing for the processor implemented in the user portable device the processed biosignal data sample segments with the associated quality indicator indicating good signal quality to the second depth learning model wherein the second depth learning model has been trained based on AFib annotations and one or more sets of AFib acquisition training data; and generating the event prediction result based on the processed biosignal data sample segments associated with a desired signal quality indicator, generating for the processor implemented in the user-portable device, and using the second depth learning model the prediction result indicative of a probability that a result in an existing segment of biosignal data samples. Method according to Claim 24 wherein generating the event prediction result indicates a probability of atrial fibrillation (AFib) in an existing segment of the biosignal data samples; and wherein the second depth learning model has been previously trained based on AFib annotations and one or more sets of AFib acquisition training data. A method of cardiac flicker (AFib) detection in a user-portable device, comprising: Receiving PPG sensor data samples from a photoplethysmographic (PPG) sensor implemented in the user-portable device; Receiving motion data samples from an inertial measurement sensor implemented in the user-portable device; Performing on a processor implemented in the user-portable device, baseline removal on the PPG sensor data samples to provide processed PPG data samples; Classifying, at the processor implemented in the user-portable device, a degree of movement of the motion data samples into a first movement indicator indicating a moderate movement and a second movement indicator indicating a strong movement; Providing to the processor implemented in the user-portable device, the processed PPG data samples and a corresponding motion indicator for a first depth learning model that has been previously trained based on a plurality of quality identifiers and one or more sets of quality data; Generating, on the processor implemented in the user-portable device, a signal quality indicator that identifies a given segment of the processed PPG data samples as having good signal quality or poor signal quality; Providing the processor implemented in the user-portable device, segments of the processed PPG data samples with associated signal quality indicator that assigns good signal quality to a second depth learning model that has been previously trained based on AFib annotations and one or more sets of AFib acquisition training data ; and Generating, in the user-portable device processor, a prediction result indicative of a likelihood that atrial fibrillation is present in a given segment of the processed PPG sensor data samples. Method according to Claim 26 , further comprising: providing an indication to the user in response to the prediction result indicating a high probability that atrial fibrillation is present in the given segment of the PPG data samples. A processor-readable, tangible and non-transitory storage medium containing heart flicker (AFib) detection commands in a user-portable device, wherein the execution of the commands by a processor device performs actions comprising: receiving PPG sensor data samples from a photoplethysmographic (PPG ) Sensor implemented in the user-portable device; Receiving motion data samples from an inertial measurement sensor implemented in the user-portable device; Performing baseline removal on the PPG sensor data samples to provide processed PPG data samples to a processor implemented in the user-portable device; Classifying, at the processor implemented in the user-portable device, a degree of movement of the motion data samples into a first movement indicator indicating a moderate movement and a second movement indicator indicating a strong movement; Providing, on the processor implemented in the user-portable device, the processed PPG signals and a corresponding motion indicator for a first depth learning model that has been previously trained based on a plurality of quality identifiers and one or more sets of quality training data; Generating, on the processor implemented in the user-portable device, a signal quality indicator that identifies a given segment of the processed PPG signal as having good signal quality or poor signal quality; Providing, on the processor implemented in the user-portable device, segments of the processed PPG signal with associated signal quality indicator that assigns good signal quality to a second learning model that previously trains based on AFib annotations and one or more sets of AFib acquisition training data has been; and generating, in the processor implemented in the user-portable device, a prediction result indicative of a likelihood that a cardiac fibrillation is present in a given segment of the biosignal data samples.

Images