JP2022502201A - Continuous monitoring of user health using mobile devices - Google Patents

Abstract

Disclosed herein are devices, systems, methods, and platforms for continuously monitoring a user's health status, such as the health status of the heart. The present disclosure is determined, for example, by either (i) a group of individuals affected by other similar factors, or (ii) the users themselves affected by other similar factors, without limitation. , Or factors that may affect health indicators to determine if a user has normal health when compared to them (referred to herein as "other factors"). A system, method for continuously monitoring health indicator data (eg, but not limited to, PPG signal, heart rate, or blood pressure) from a user device in combination with the corresponding (temporally) data associated with. , Devices, software, and platforms.

Description

Mutual reference to related applications This application claims the benefit of US Provisional Application No. 61 / 915,113 filed December 12, 2013, which is incorporated herein by reference, and is now August 2016. A continuation of US Patent No. 9,420,956 issued on March 23, US Application No. 14 / 569,513 filed on December 12, 2014, and now a US patent issued on February 21, 2017. A continuation of U.S. Application No. 14 / 730,122 filed on June 3, 2015, No. 9,572,499, and a partial continuation of U.S. Application No. 15 / 393,077 filed on December 28, 2016. U.S. Application No. 16 / 153,345 filed October 5, 2018, U.S. Provisional Application No. 61 / 953,616 filed March 14, 2014, 2014. US Provisional Application No. 61 / 969,019 filed on March 21, 2014, US Provisional Application No. 61 / 970,551 filed on March 26, 2014, and the application is incorporated herein by reference, June 2014. It claims the priority of US Provisional Application No. 62 / 014,516 filed on 19th March. This application is also incorporated herein by reference in its entirety, US Provisional Application No. 62 / 569,309 filed October 6, 2017, and its entire contents by reference. It claims the benefit of US Provisional Application No. 62,589,477 filed on November 21, 2017.

Indicators of an individual's physiological health (“health indicators”), such as, but not limited to, heart rate, heart rate variability, blood pressure, and ECG (electrocardiogram), to measure health indicators. Can be measured or calculated at any individual time point from the data collected in. Often, the values of health indicators at a particular point in time, changes over time, provide information about an individual's health. For example, an ECG that clearly indicates low or high heart rate or blood pressure, or myocardial ischemia may indicate the need for immediate intervention. However, readings of these indicators, a series of readings, or changes in readings over time may provide information that is not recognized as requiring attention by the user, or even a healthcare professional. be.

For example, arrhythmias may occur continuously or intermittently. Continuously occurring arrhythmias may be most clearly diagnosed from an individual's electrocardiogram. Since continuous arrhythmias are always present, ECG analysis can be applied at any time to diagnose arrhythmias. ECG can also be used to diagnose intermittent arrhythmias. However, intermittent arrhythmias can be asymptomatic and / or, by definition, continuous, presenting the challenge of applying diagnostic techniques when an individual is experiencing an arrhythmia. Therefore, the actual diagnosis of continuous arrhythmia is more difficult the more notorious it is. This particular difficulty is exacerbated by asymptomatic arrhythmias, which account for nearly 40% of arrhythmias in the United States. Boriani G. and Pettorelli D., Atrial Fibrillation Burden and Atrial Fibrillation type: Clinical Significance and Impact on the Risk of Stroke and Decision Making for Long-term Anticoagulation, Vascul Pharmacol., 83: 26-35 (August 2016), Page 26.

There are sensors and mobile electronic technologies that enable frequent or continuous monitoring and recording of health indicators. However, the capabilities of these sensor platforms often exceed the capabilities of traditional medicine to interpret the data they generate. The physiological importance of health indicator parameters such as heart rate is often clearly defined only in certain medical situations, for example, heart rate is usually another that can affect health indicators. Evaluated as a single scalar value out of context from the data / information. Resting heart rates in the range of 60-100 beats / minute (BPM) may be considered normal. The user can generally manually measure the user's resting heart rate once or twice a day.

Mobile sensor platforms (eg, mobile blood pressure cuffs, mobile heart rate monitors, or mobile ECG devices) also have, but are not limited to, other data about the user such as activity level, position, and environmental parameters such as temperature, pressure, and location. Health indicators (eg, heart rate) can be continuously monitored while being acquired simultaneously, for example, measurements can be generated every 1 or 5 seconds. During the 24-hour period, this can result in thousands of independent health index measurements. There is relatively little data or medical consensus on what a "normal" series of thousands of measurements will look like, as opposed to once or twice daily measurements.

Devices currently used to continuously measure user / patient health indicators range from bulky, invasive and inconvenient to simple wearable or handheld mobile devices. Currently, these devices do not provide the ability to effectively use data to continuously monitor a person's health. It is up to the user or healthcare professional to evaluate the health index in the light of other factors that may affect these health indicators in order to determine the health status of the user.

US Provisional Application No. 61 / 872,555 U.S. Pat. No. 9,351,654 U.S. Pat. No. 9,247,911 U.S. Pat. No. 9,254,095 U.S. Pat. No. 8,509,882 U.S. Patent Application Publication No. 2015/0018660 U.S. Patent Application Publication No. 2015/0297134 U.S. Patent Application Publication No. 2015/0320328

Boriani G. and Pettorelli D., Atrial Fibrillation Burden and Atrial Fibrillation type: Clinical Significance and Impact on the Risk of Stroke and Decision Making for Long-term Anticoagulation, Vascul Pharmacol., 83: 26-35 (August 2016), Page 26

The particular features described herein are specifically described in the appended claims. For a better understanding of the features and advantages of the disclosed embodiments, refer to the following detailed description revealing exemplary embodiments in which the principles described herein are utilized and the accompanying drawings thereof. Obtained by.

It is a figure which shows the convolutional neural network which can be used according to some embodiments described herein. It is a figure which shows the convolutional neural network which can be used according to some embodiments described herein. It is a figure which shows the recurrent neural network which can be used according to some embodiments described herein. It is a figure which shows the recurrent neural network which can be used according to some embodiments described herein. It is a figure which shows the alternative recurrent neural network which can be used according to some embodiments described herein. It is a figure which shows the virtual data plot for demonstrating the application of some embodiments described herein. It is a figure which shows the virtual data plot for demonstrating the application of some embodiments described herein. It is a figure which shows the virtual data plot for demonstrating the application of some embodiments described herein. It is a diagram showing an alternative recurrent neural network according to some embodiments described herein and a virtual plot used to illustrate some of these embodiments. It is a diagram showing an alternative recurrent neural network according to some embodiments described herein and a virtual plot used to illustrate some of these embodiments. It is a diagram showing an alternative recurrent neural network according to some embodiments described herein and a virtual plot used to illustrate some of these embodiments. It is a diagram showing an alternative recurrent neural network according to some embodiments described herein and a virtual plot used to illustrate some of these embodiments. It is a diagram showing an alternative recurrent neural network according to some embodiments described herein and a virtual plot used to illustrate some of these embodiments. It is a figure which shows the developed recurrent neural network by some embodiments described herein. It is a figure which shows the system and the device by some embodiments described herein. It is a figure which shows the system and the device by some embodiments described herein. It is a figure which shows the method by some embodiments described in this specification. FIG. 6 illustrates a method according to some embodiments described herein and a virtual plot of heart rate vs. time to demonstrate one or more embodiments. FIG. 6 illustrates a method according to some embodiments described herein and a virtual plot of heart rate vs. time to demonstrate one or more embodiments. It is a figure which shows the method by some embodiments described in this specification. It is a figure which shows the virtual data plot for demonstrating the application of some embodiments described herein. It is a figure which shows the system and the device by some embodiments described herein.

Large amounts of data, the complexity of interconnections between health indicators and other factors, and limited clinical guidance are abnormalities in continuous and / or mobility sensor data through specific rules based on traditional medical practice. May limit the effectiveness of any monitoring system that attempts to detect. The embodiments described herein utilize a predictive machine learning model to be unsupervised, either alone from the time sequence of health index data or in combination with other factor data (as defined herein). Includes devices, systems, methods, and platforms that can detect anomalies in.

Atrial fibrillation (AF or AFib) is found in 1-2% of the general population, and the presence of AF increases the morbidity and risk of adverse outcomes such as stroke and heart failure. Boriani G. and Pettorelli D., Atrial Fibrillation Burden and Atrial Fibrillation type: Clinical Significance and Impact on the Risk of Stroke and Decision Making for Long-term Anticoagulation, Vascul Pharmacol., 83: 26-35 (August 2016), Page 26. AFib in many, estimated to be around 40% of AF patients, can be asymptomatic, and these asymptomatic patients have the same risk profile for stroke and heart failure as symptomatic patients. Please refer to the same document. However, symptomatic patients can take proactive measures, such as taking anticoagulants or other medications, to reduce the risk of negative consequences. The use of implantable electrical devices (CIED) can detect asymptomatic AF (so-called silent AF or SAF) and the duration of AF in the patient. The same document. From this information, the time these patients spend in AF, ie the AF burden, can be determined. The same document. AF burdens of more than 5-6 minutes, especially more than 1 hour, are associated with a significant increased risk of stroke and other negative health outcomes. The same document. Therefore, the ability to measure AF burden in asymptomatic patients can lead to early interventional treatment and reduce the risk of negative health outcomes associated with AF. The same document. Detection of SAF is difficult and typically requires some form of continuous monitoring. Currently, continuous monitoring of AF requires bulky, sometimes invasive, and expensive devices, and such monitoring requires the monitoring and review of advanced medical professionals.

Many devices continuously acquire data to provide measurements or calculations of health indicator data, such as, but not limited to, FitBit®, Apple Watch®, Polar®. , Smartphones, tablets, etc. fall into the category of wearable devices and / or mobile devices. Other devices include permanent or semi-permanent devices (eg, Holter) on or within the user / patient, other devices may be mobile by being in a cart, within the hospital. May include larger devices. However, this measured data is rarely used except to monitor it regularly on the display or set simple data thresholds. Even by trained medical professionals, observations of the data can often appear normal, with one major exception being those with easily discernible acute symptoms. It is very difficult and in fact difficult for healthcare professionals to continuously observe health indicators to observe anomalies and / or trends in the data that may indicate something more serious. Is impossible.

As used herein, a platform is one or more customized software applications (or "applications" configured to interact with each other locally or over a distributed network that includes the cloud and the Internet. )including. The platform applications described herein are configured to collect and analyze user data and may include one or more software models. In some embodiments of the platform, the platform comprises one or more hardware components (eg, one or more sensing devices, processing devices, or microprocessors). In some embodiments, it is configured to work with one or more devices and / or one or more systems. That is, the device described herein is configured in some embodiments to run an application on a platform using an embedded processor, and in some embodiments the platform is one of the platforms. Or utilized by a system with one or more computing devices that interact with or run multiple applications.

The present disclosure is determined, for example, by either (i) a group of individuals affected by other similar factors, or (ii) the user himself / herself affected by other similar factors, without limitation. Factors that may affect health indicators (referred to herein as "other factors") to determine if a user has normal health status or when compared to them. ) Related (eg, temporally) in combination with one or more health indicators (eg, but not limited to, PPG signal, heart rate, or blood pressure) from the user device. Describes systems, methods, devices, software, and platforms for continuous monitoring. In some embodiments, the measured health index data is input to a trained machine learning model, either alone or in combination with other factor data, where the machine learning model is such that the user's measured health index is healthy. Determines the probability that it is considered to be within the range, and if not, notifies the user to that effect. Users who are not within the healthy range are likely to be experiencing health events that guarantee high fidelity information to confirm the diagnosis, such as arrhythmias that can be symptomatic or asymptomatic. May increase. The notification can be in the form, for example, asking the user to get an ECG. Other high fidelity measurements may require blood pressure, pulse oximeters, to name two, and ECG is just one example. High fidelity measurement, in this embodiment the ECG is an algorithm for making notifications or diagnoses (collectively referred to herein as "diagnosis", recognizing that only a physician can make a diagnosis). And / or can be evaluated by a medical professional. In the example of ECG, the diagnosis can be AFib, or any number of other well-known conditions diagnosed using ECG.

In a further embodiment, the diagnosis is used to label a low fidelity data sequence (eg, heart rate or PPG) that may include data sequences of other factors. This low fidelity data sequence with the high fidelity diagnostic label is used to train high fidelity machine learning models. In these further embodiments, the training of the high fidelity machine learning model can be trained by unsupervised learning or can be updated from time to time with new training examples. In some embodiments, the user's measured low fidelity health index data sequence, and optionally the (temporal) corresponding data sequence of other factors, is diagnosed trained with a high fidelity machine learning model. It is input to a trained high fidelity machine learning model to determine the probability and / or prediction that the user is experiencing or experiencing the condition. This probability can include the probability of when the event starts and ends. Some embodiments may calculate, for example, the amount of atrial fibrillation (AF) burden of the user, or the amount of time the user experiences AF over time. Previously, AF burden could only be determined using awkward and expensive Holter or an embedded continuous ECG monitoring device. Accordingly, some embodiments described herein continually capture health index data (eg, but not limited to, PPG data, blood pressure data, and heart rate data) obtained only from the device worn by the user. You can continuously monitor your health and notify you of changes in your health, either by monitoring or in combination with corresponding data about other factors. As used herein, "other factors" include those that may affect health indicators and / or data that represent health indicators (eg, PPG data). .. These other factors, such as, but not limited to, temperature, altitude, exercise level, weight, gender, diet, standing, sitting, falls, lying down, weather, and BMI, to name a few. Can include various factors such as. In some embodiments, mathematical or empirical models may be used rather than machine learning models to determine when to notify the user to obtain high fidelity measurements, which are high fidelity measurements. Can then be analyzed and used to train the high fidelity machine learning models described herein.

Some embodiments described herein include the operation of receiving a primary time sequence of health indicator data and one or more of the other factor data that corresponds temporally to the primary time sequence of the health indicator data. Actions and filtering, which are actions that optionally receive the next time sequence, and the second time sequence may come from a sensor or from an external data source (eg, via a network connection, computer API, etc.). Operations that provide primary and secondary time sequences to preprocessors that can perform operations such as caching, averaging, time matching, buffering, upsampling, and downsampling on data, and primary sequences in future time. Machine learning at a particular time t, with the behavior of providing a time sequence of data to a machine learning model trained and / or configured to utilize the values of the primary and secondary time sequences to predict the next value. There is a difference between the behavior of comparing the predicted primary time sequence values generated by the model with the measured values of the primary time sequence at time t and the predicted future time sequence and the measured time sequence. If the value or criterion is exceeded, the user's anomaly can be detected in an unsupervised manner by warning or prompting the user to take action.

Therefore, in some embodiments described herein, the observed behavior of a primary sequence of physiological data in response to a secondary sequence of data and / or observed over time trains the model. Detects cases where the training examples used for are different from those given and predicted. The system can act as an anomaly detector if the training examples are collected from a normal individual or from data previously classified as normal for a particular user. If the data was simply collected from a particular user without any other classification, the system would detect changes in the health indicator data for which the primary sequence was measured compared to the time the training data was captured. It can function as a detector.

Described herein is a healthy population in which the measured health index data (primary sequence) of a user under the influence of another factor (secondary sequence) is under the influence of other factors as well. To predict or determine the probability of being out of the normal range for (ie, the global model) or for that particular user (ie, the personalized model) under the influence of other similar factors. , Software platforms, systems, devices, and methods for generating and using trained machine learning models. In some embodiments, the user previously used only low fidelity health index data to generate different trained high fidelity machine learning models capable of predicting or diagnosing anomalies or events. Obtained low fidelity User health indicator data may be prompted to obtain additional measured high fidelity data that can be used to label such anomalies, such anomalies are typically high fidelity. It is identified or diagnosed only using degree data.

Some embodiments described herein include the action of entering user health index data and, optionally, the corresponding (temporal) data of other factors into a trained machine learning model. A trained machine learning model, which may include behavior, predicts the probability distribution of the user's health index data or health index data in future time steps. The predictions in some embodiments are compared to the user's measured health index data in the time step of the prediction, and if the absolute value of the difference exceeds the threshold, the user is out of the normal range of the user's health index data. You will be notified that This notification may include, in some embodiments, a diagnosis or instruction to do something, eg, to obtain additional measurements, or to contact a medical professional, without limitation. In some embodiments, health index data from a population of healthy people and (temporally) corresponding data of other factors are used to train machine learning models. The other factors in the training example used to train the machine learning model do not have to be the mean of the population, and the data for each of the other factors is in the collection of health index data for the individual in the training example. It will be understood that it corresponds in time.

Some embodiments receive discrete data points in time and predict the discrete data points in future time from the input, followed by discrete measurement inputs in future time and predictions in future time. Described as determining whether the loss between values exceeds the threshold. One of skill in the art will readily appreciate that input data and output predictions can take forms other than discrete data points or scalars. For example, but not limited to, health index data sequences (also referred to herein as linear sequences) and other data sequences (also referred to herein as secondary sequences) can be divided into time segments. Those skilled in the art will recognize that the way data is segmented is a matter of design choice and can take many different forms.

In some embodiments, a health index data sequence (also referred to herein as a primary sequence) and another data sequence (also referred to herein as a secondary sequence) are divided into two segments, prior to a particular time t. Divide into the past, which represents all data, and the future, which represents all data after time t. These embodiments include health indicator data sequences for past time segments, and all other data sequences for past time segments, with the most likely future segments (or possible future) of health indicator data. Input to a machine learning model configured to predict the distribution of segments). Alternatively, these embodiments include health indicator data sequences for past time segments, all other data sequences for past time segments, and other data sequences from future segments, most of the health indicator data. Enter into a machine learning model that is configured to predict the probable future segments (or the distribution of the probable future segments). The predicted future segment of health indicator data is compared to the user's measured health indicator in the future segment to determine the loss and whether the loss exceeds the threshold, resulting in a loss. If the threshold is exceeded, some action will be taken. The action is, for example, not limiting, but notifying the user to get additional data (eg, ECG or blood pressure), notifying the user to contact a healthcare professional, or getting additional data. It may include actions that trigger automatically. Automatic acquisition of additional data is, for example, not limited to ECG acquisition via sensors operably coupled (wired or wirelessly) to the user-worn computing device, or the user's wrist or other suitable body part. It may include blood pressure through a mobile cuff that is around and coupled to the computing device worn by the user. A segment of the data contains a single data point, many data points over a time period, the mean of these data points over a time period, and the mean contains the true mean, median, or mode. obtain. In some embodiments, the segments can overlap in time.

In these embodiments, the observed behavior or measurement of the sequence of health index data with respect to the passage of time affected by the sequence of data of the corresponding other factors (in time) is expected from the training examples. Detecting different cases, training examples are collected under similar other factors. These embodiments, when training examples are collected from healthy individuals under similar other factors, or from data previously classified as healthy for a particular user under similar other factors. Act as anomaly detectors from healthy populations or from specific users, respectively. If the training examples were simply collected from a particular user without any other classification, these embodiments would be a conversion in the health index at the time of measurement compared to the time the training examples were collected for the particular user. Functions as a change detector to detect.

Some embodiments described herein continuously monitor a person's health index under the influence of one or more other factors and as health under the influence of similar other factors. Use machine learning to assess whether a person is healthy, taking into account the classified population. As will be easily understood by those skilled in the art, several different machine learning algorithms or models (including, but not limited to, Bayesian, Markov, Gaussian processes, clustering algorithms, generative models, kernels, and neural network algorithms) are available in the book. It can be used without exceeding the scope described in the specification. As will be appreciated by those of skill in the art, typical neural networks, as an example, use one or more layers of a nonlinear activation function to predict the output for a received input, not limited to the input. It may contain one or more hidden layers in addition to the layers and output layers. The output of each hidden layer in some of these networks is used as input to the next layer in the network. Examples of neural networks include, but are not limited to, generated neural networks, convolutional neural networks, and recurrent neural networks, as examples.

Some embodiments of the health monitoring system monitor an individual's heart rate and activity data as low fidelity data (eg, heart rate or PPG data) and use high fidelity data (eg, ECG data). Detects normally detected conditions (eg AFib). For example, an individual's heart rate can be provided by sensors at continuous or discrete intervals (such as every 5 seconds). Heart rate can be determined based on PPG, pulse oximetry, or other sensors. In some embodiments, activity data can be generated as steps, the amount of perceived movement, or other data points indicating the level of activity. Low fidelity (eg, heart rate) and activity data can then be input to the machine learning system to determine predictions of high fidelity results. For example, machine learning systems may use low fidelity data to predict arrhythmias, or other signs of a user's heart health. In some embodiments, the machine learning system may use the inputs of segments of data inputs to determine predictions. For example, one hour of activity level data and heart rate data may be input to the machine learning system. The system can then use the data to generate predictions of conditions such as atrial fibrillation. Various embodiments of the invention are discussed more fully below.

Referring to FIG. 1A, a trained convolutional neural network (CNN) 100 (an example of a feedforward network) puts input data 102 (eg, a picture of a boat) into a convolutional layer (also known as a hidden layer) 103 and sets a series. The trained weight or filter 104 is applied to the input data 103 at each of the convolutional layers 103. The output of the first convolution layer is an activation map (not shown) that is the input to the second convolution layer, to which a trained weight or filter (not shown) is applied. The output of the subsequent convolutional layer results in an activation map that represents the increasingly complex features of the input data to the first layer. After each convolution layer, a non-linear layer (not shown) is applied to introduce non-linearity into the problem, and the non-linear layer can include tanh, sigmoid, or ReLU. In some cases, a pooling layer (not shown) is applied after the non-linear layer, which is also called a downsampling layer and is basically applied to the input with filters and strides of the same length. Outputs the maximum number in all subregions that the filter convolves. Other options for pooling are average pooling and L2-norm pooling. The pooling layer reduces the spatial dimensions of the input volume, reduces computational costs and controls overfitting. The last layer of the network is the fully connected layer, which receives the output of the last convolution layer and outputs an n-dimensional output vector representing the amount to be predicted, eg, the probability of image classification, 20. Outputs% cars, 75% boats, 5% buses, and 0% bicycles, ie results in a predicted output of 106 (O *), for example, this could be a picture of a boat. expensive. The output can be a data point of a scalar value predicted by the network, for example a stock price. The trained weight 104 can be different for each of the convolution layers 103, as described more fully below. To achieve this real-world prediction / detection (eg, it is a boat), the neural network needs to be trained against known data inputs or training examples, resulting in a trained CNN100. Occurs. To train the CNN100, many different training examples (eg, many pictures of boats) are input to the model. Those skilled in the art of neural networks
It is well understood that the above explanation provides a somewhat simpler view of CNNs to provide some context for the current discussion, and the application of any CNN alone or in combination with other neural networks, It will be fully appreciated that they are equally applicable and are within the scope of some of the embodiments described herein.

FIG. 1B shows the training CNN 108. In FIG. 1B, the convolutional layer 103 is shown as the individual hidden convolutional layers 105, 105'up to the ^{convolutional layer 105 n-1, and the final nth layer is the fully connected layer.} It will be understood that the last layer is two or more fully connected layers. Training example 111 is input to the convolution layer 103, a nonlinear activation function (not shown) and weights 110, 110'to 110 ⁿ are applied in series to training example 111, and the output of any hidden layer is as follows: It is input to the layer and continues until the last ^{nth fully connected layer 105 n produces output 114.} The output or prediction 114 is compared to training example 111 (eg, a picture of a boat) and results in a difference 116 between the output or prediction 114 and training example 111. If the difference or loss 116 is less than any preset loss (eg, the output or prediction 114 predicts that the object is a boat), the CNN is considered converged and trained. If the CNN is not converged, the backpropagation technique is used and the weights 110 and 110'to 110 ⁿ are updated depending on how close the prediction is to the known input. Those of skill in the art will appreciate that methods other than backpropagation can be used to adjust the weights. A second training example (eg, a different picture of the boat) is entered, the process is repeated again with the updated weights, and the updated weights are then the nth training example (eg, the nth). It will be updated again until the boat's nth photo) is entered. This is repeated many times with the same n training examples until the convolutional neural network (CNN) is trained or converges to the correct output for a known input. When the CNN 108 is trained, the weights 110, 110'to 110 ⁿ are used in the fixed and trained CNN 100, and these weights are the weights 104 as shown in FIG. 1A. As described, there are different weights for each convolution layer 103 and each of the fully connected layers. The trained CNN100 or model is then supplied with image data to determine or predict what it has been trained to predict / identify (eg, a boat), as described above. Any trained model, CNN, RNN, etc. can be further trained with additional training examples or with the predictive data output by the model then used as training examples, ie weight correction. Can be allowed. Machine learning models can be trained "offline" and, for example, can be trained on a computing platform separate from the platform on which the trained model is used / executed and then transferred to that platform. Alternatively, the embodiments described herein may periodically or continuously update the machine learning model based on newly acquired training data. This updated training can occur on another computing platform that delivers the updated training model to a platform that uses / runs the retrained model over a network connection, or the training / retraining / updating process. Can occur on the platform itself as new data is acquired. Those of skill in the art will appreciate that CNNs are applicable to data in fixed sequences (eg, photographs, letters, words, etc.), or time sequences of data. For example, ordered health index data and other factor data can be modeled using CNN. Some embodiments utilize feedforwards, CNNs with skip connections, and Gaussian mixed model outputs to determine probability distributions for predicted health indicators such as heart rate, PPG, or arrhythmias.

Some embodiments may utilize neural networks of other types and configurations. The number of convolutional layers can be increased or decreased as well as the number of fully connected layers. In general, the optimal number and proportion of convolutional layers and fully connected layers can be set experimentally by determining which configuration gives the best performance in a given data set. The number of convolution layers is reduced to 0, leaving a fully coupled network. The number of convolution layers and the width of each filter can also be increased or decreased.

The output of the neural network can be a single scalar value that corresponds to an accurate prediction for the first-order time sequence. Alternatively, the output of the neural network can be a logistic regression in which each category corresponds to a particular range or class of first-order time sequence values, which is any number of alternative outputs easily understood by those of skill in the art.

The use of Gaussian mixture model output in some embodiments is intended to constrain the network to learn a well-formed probability distribution and improve generalization in restricted training data. The use of multiple elements in some embodiments in a Gaussian mixture model is intended to allow the model to learn a multimodal probability distribution. Machine learning models that combine or aggregate the results of different neural networks can also be used and the results can be combined.

Machine learning models with updatable memories or states from previous predictions for application to subsequent predictions are another technique for modeling sequence data. In some specific embodiments described herein, recurring neural networks are utilized. With reference to the example in Figure 2A, a diagram of a trained recurrent neural network (RNN) 200 is shown. The trained RNN200 has an updatable state (S) 202 and a trained weight (W) 204. The input data 206 is input to the state 202, where the weight (W) is applied and the prediction 206 (P *) is output. In contrast to a linear neural network (eg CNN100), state 202 is updated based on the input data, thereby serving as a memory from the previous state for the next prediction with the next data in turn. .. Updating the state gives the RNN a circular or looping mechanism. To better demonstrate, Figure 2B shows the deployed and trained RNN200 and its applicability to sequence data. When expanded, the RNN appears to be similar to the CNN, but in the unexpanded RNN, each of the seemingly similar layers is updated, with the same weights applied at each iteration of the loop. Appears as a single layer with a state. A single layer is depicted here for clarity, but one of ordinary skill in the art will recognize that the single layer itself can have sublayers. The input data (I _t ) 208 at time t is _{input to the state (S t} ) 210 at time t, and the trained weight 204 is applied into the _{cell (C t) 212 at time t.} The output of C _t 212 is predicted at time step t + 1.

214, and updated state S _{t + 1} 216. Similarly, at C _{t + 1} 220, I _{t + 1} 218 is _{input to St + 1} 216, the same trained weight 204 is applied, and the output of _{C t + 1 220 is}

It is 222. As mentioned above, S _{t + 1} is _{updated from S t} , so S _{t + 1} has a memory from _{S t} from the previous time step. For example, without limitation, this memory may include previous health index data or previous other factor data from one or more previous time steps. This process continues for n steps, It _{+ n} 224 is _{entered in St + n} 226 and the same weight 204 is applied. The output of cell C _{t + n is predicted}

Is. In particular, the state is updated from the previous time step, giving the RNN the advantage of memory from the previous state. This property makes RNNs an alternative option for making predictions in sequence data for some embodiments. However, as described above, there are other suitable machine learning techniques for making such predictions in sequence data, including CNNs.

Like a CNN, an RNN can process a string of data as input and output the predicted string of data. A simple way to illustrate this aspect of using RNNs is to use an example of natural language prediction. Use the phrase The sky is blue. Word (ie, data) columns have context. Therefore, when the state is updated, the data column is updated at each iteration, providing a context for predicting blue. As just described, RNNs have storage components to assist in making predictions in sequence data. However, the memory of the updated state of the RNN, similar to short-term memory, can be limited to the extent that it can be looked back. Similar to long-term memory, when predicting sequence data for which longer reflection is desired, the RNN tweaks just described can be used to achieve this. A sentence in which the word to be predicted is unclear from the immediately preceding or surrounding words, Mary speaks fluent French is another simple example to explain. It is unclear from the previous word that French is the correct prediction, that is, it is unclear which language it is, just because one language is the correct prediction. The correct prediction may be in the context of words separated by gaps larger than a single word string. Long Short Term Memory (LSTM) networks are a special kind of RNN that can learn these (more) long term dependencies.

As described above, RNNs have a relatively simple repeating structure, for example, they may have a single layer with a non-linear activation function (eg, tanh or sigmoid). LSTMs also have a chain-like structure, but (eg) have four neural network layers instead of one. These additional neural network layers give the LSTM the ability to remove or add information to the state by using a structure called a cell gate. The same document. Figure 3 shows cell 300 for an LSTM RNN. Line 302 represents the state of the cell (S) and can be regarded as an information highway, and it is relatively easy for information to flow along the state of the cell where it does not change. The same document. Cellgates 304, 306, and 308 determine how much information is allowed through the state or along the information highway. Cell gate 304, first, how much more information is removed from the cell state S _t, determining the so-called forgetting gate layer. The same document. Cell gates 306 and 306'then determine what information is added to the cell state, and cell gates 308 and 308' predict.

To determine what is output from the cell state. The information highway or cell state is now updated to _{cell state S t + 1 for use in the next cell.} LSTMs allow RNNs to have more permanent or (longer) long-term memory. LSTM is RNN-based machine learning in that output forecasting takes into account contexts that are separated from the input data by more space or time than simpler RNN structures, depending on how the data is ordered. Provides additional benefits to the model.

In some embodiments that utilize RNNs, the primary and secondary time sequences may not be provided to the RNN as a vector at each time step. Instead, the RNN may be provided with only the current values of the primary and secondary time sequences, along with future values or aggregate functions of the secondary time sequences within the prediction interval. In this way, the RNN uses a persistent state vector to hold information about previous values for use in making predictions.

Machine learning is well suited for continuous monitoring of one or more criteria for identifying large or small anomalies or trends in input data compared to the training examples used to train the model. Therefore, some embodiments described herein input user health index data and optionally other factor data into a trained machine learning model, where the trained machine learning model is healthy. Predict what a good person's health index data will look like in the next time step, and compare the prediction with the user's measured health index data in a future time step. If the absolute value of the difference (eg, the loss described below) exceeds the threshold, the user is informed that the user's health index data is not within the normal or healthy range. The threshold is a number set by the designer and, in some embodiments, can be changed by the user to allow the user to adjust the notification sensitivity. The machine learning models of these embodiments can be trained in health indicator data alone or in combination with corresponding other factor data (in time) from a population of healthy people, or to meet the design needs of the model. Can be trained in other training examples.

Data from health indicators such as heart rate data is sequence data, more specifically time sequence data. Heart rate can be measured, for example, by a number of different methods, but not limited to, such as by measuring an electrical signal from a chest strap or being derived from a PPG signal. In some embodiments, the heart rate derived from the device is obtained and each data point (eg, heart rate) is generated at approximately equal intervals (eg, 5 seconds). However, in some cases, in other embodiments, the derived heartbeat is derived, for example, because the data required for derivation is unreliable (eg, the PPG signal is unreliable because the device has moved or because of light pollution). Numbers are not provided in approximately equal time steps. The same is true for obtaining a quadratic sequence of data from a motion sensor or other sensor used to collect other factor data.

The raw signal / data (an electrical signal from the EEC, chest strap, or PPG signal) itself is a time sequence of data that can be used according to some embodiments. For clarification purposes, this description, but not a limitation, uses PPG to refer to data representing health indicators. One of ordinary skill in the art can use any form of data regarding health indicators, raw data, waveforms, or numbers derived from raw data or waveforms according to some embodiments described herein. Will be easily recognized.

Machine learning models that can be used in the embodiments described herein include, but are not limited to, Bayesian, Markov, Gaussian processes, clustering algorithms, generative models, kernels, and neural network algorithms. Some embodiments utilize machine learning models based on trained neural networks, others utilize recurrent neural networks, and additional embodiments utilize LTSM RNNs. For purposes of clarity, but not limited to, recurrent neural networks are used to illustrate some embodiments herein.

FIGS. 4A-4C show a virtual plot of PPG over time (FIG. 4A), a virtual plot of steps over time (FIG. 4B), and a virtual plot of temperature over time (FIG. 4C). PPG is an example of health index data, and steps, activity levels, and temperature are examples of other factor data on other factors that may affect health index data. As will be appreciated by those of skill in the art, other data may, but are not limited to, be obtained from any of many known sources, including, but not limited to, GPS data, scales, user input, etc. It may include temperature, activity (running, walking, sitting, cycling, falling, climbing stairs, walking, etc.), BMI, weight, height, age, etc. The first dotted line running vertically through all three plots represents the time t at which user data was acquired for input into a trained machine learning model (discussed below). The hashed plot line in FIG. 4A represents the predicted or likely output data 402, and the solid line 404 in FIG. 4A represents the measured data. FIG. 4B is a virtual plot of the user's steps at various times, and FIG. 4C is a virtual plot of temperature at various times.

5A-5B are of the trained recurrent neural network 500 for receiving the input data shown in FIGS. 4A-4C, namely PPG (P), steps (R), and temperature (T). A schematic diagram is shown. It is emphasized again that these input data (P, R, and T) are just examples of health index data and other factor data. It is also understood that data on more than one health indicator can be entered and predicted, more or less other factor data can be used, and the choice depends on what the model is designed for. Yeah. It will be further understood by those skilled in the art that other factor data will be collected in a timely manner with the collection or measurement of health indicator data. In some cases, other factor data, such as body weight, remains relatively constant over a particular time period.

Figure 5A shows the trained neural network 500 as a loop. P, T, and R are input to state 502 of RNN500, where weight W is applied, and RNN500 outputs the predicted PPG504 (P ^* ). In step 506, the difference PP ^* (ΔP ^* ) is calculated, and in step 508 ^{it is determined whether | ΔP *} | is greater than the threshold. If so, step 510 notifies / warns the user that the user's health index is outside the predicted boundaries / thresholds for a normal or healthy person. Warnings / notifications / detections are not limited, for example, doctor consultation / consultation suggestions, simple notifications such as tactile feedback, requests to take additional measurements such as ECG, or simple notes without recommendations, Or any combination thereof. If | ΔP ^* | is less than or equal to the threshold, step 512 does nothing. In both step 510 and step 512, the process is repeated with new user data in the next time step. In this embodiment, the state is updated following the output of the predicted data, and the predicted data may be used when updating the state.

In another embodiment not shown, a primary sequence of heart rate data (eg, derived from a PPG signal) and a secondary sequence of other factor data are provided in a trained machine learning model and trained machine learning. The model can be an RNN, CNN, other machine learning model, or a combination of models. In this embodiment, the machine learning model is
_{A. A vector (V H} ) of length 300 of the last 300 health index samples (eg, heart rate per minute) containing it up to any health index data at time t,
With the latest other factor data at approximately time for each sample in _{BV H , for example, at least one vector (V O} ) of length 300, including steps.
_{C. A vector (V TD} ) of length 300, in which the entry V _DT (i) at index i contains the time difference between the time stamps of the _{health index samples V H} (i) and V _{H (i-1).}
A scalar prediction interval representing the mean other factor rate (eg, step rate) measured over a time period from Dt to t + τ. The other factor rate O _rate (eg, not limited, but step rate), where τ is. For example, it is not limited to 2.5 minutes, and _{is configured to receive the scalar prediction interval and other factor rate O rate} , which is the future prediction interval, as an input at the reference time t.

The output of this embodiment can be, for example, a probability distribution that characterizes the predicted heart rate measured over a time period from t to t + τ. In some embodiments, the machine learning model is trained with a training example that includes a continuous time sequence of health index data and a sequence of other factor data. In one alternative embodiment, the notification system assigns a time stamp to each predicted health index (eg, heart rate) distribution of t + τ / 2, thus concentrating the predicted distribution within the prediction interval (τ). The notification logic, in this embodiment, then _{considers all the samples in a sliding window (W) of length W L} = 2 * (τ) or 5 minutes in this example, three parameters,
1. Average value of all health index sequence data in the time window

When,
2. Predicted time stamps fall within the time window Average of all model predictions for health indicators

When,
3. Median root mean square of each predicted health index distribution in the time window

And calculate, here,
4. In one embodiment

or

If, a notification is generated, where Ψ is the threshold.

In this embodiment, a warning is generated if the measured health index exceeds a specific multiple of the standard deviation from the mean of the predicted health index values in a particular window W. Windows W can be applied in a sliding manner over a sequence of measured and predicted health indicators, with each window overlapping the designer-specified portion, eg, the previous window by 0.5 minutes.

Notifications can take any number of different forms. For example, the user may be informed to obtain ECG and / or blood pressure, but not limited, and the computing system (eg, wearable) may be instructed to automatically obtain ECG and / or blood pressure (eg). You can notify the user to get a doctor's diagnosis, or simply notify the user that the health index is not normal.

_{The choice of V DT} as an input to the model in this embodiment is intended to allow the model to utilize the information contained within the variable intervals between the health indicator data in _{V H.} Variable intervals can result from algorithms that derive health index data from inconsistent raw data. For example, a heart rate sample is generated by the Apple Watch algorithm only if it has raw PPG data that is reliable enough to output a reliable heart rate value, which is an irregular time between heart rate samples. The result is a gap. Similarly, this embodiment has other factor data having the same length as the other vector in order to handle different irregular sample rates between the primary sequence (health index) and the secondary sequence (other factor). Use the vector (V _O ) for. The secondary sequence is, in this embodiment, remapped or interpolated at the same time point as the primary time sequence.

Moreover, in some embodiments, the composition of the data from the secondary time sequence presented as input to the machine learning model from a future predicted time interval (eg, after t) can be modified. In some embodiments, a single scalar value containing the average other factor data rate over the prediction interval can be modified with multiple scalar values, eg, one for each secondary time sequence. Alternatively, a vector of values may be used over the prediction interval. In addition, the prediction interval itself can be adjusted. For example, shorter prediction intervals can provide a faster response to changes and improved detection of events with a (shorter) basic timescale, but against interference from noise sources such as motion artifacts. It can also be more sensitive.

Similarly, the output prediction of the machine learning model itself does not have to be a scalar. For example, some embodiments may generate a time series of predictions for multiple time t within the time interval between t and t + τ, and the warning logic may make each of these predictions within the same time interval. Can be compared with the measured value of.

In this preceding embodiment, the machine learning model itself may include, for example, a seven-layer feedforward neural network. The first three layers can be a convolutional layer containing 32 kernels, each with a kernel width of 24 and a stride of 2. The first layer may have arrays V _H , V _O , and V _{T D} in three channels as inputs. The last four layers can be fully connected layers and all utilize the hyperbolic tangent activation function except the last layer. The output of the third layer can be flattened into one array for input to the first fully coupled layer. The last layer outputs 30 values (mean, variance, and weight for each mixture) parameterized by a Gaussian mixture model with 10 mixtures. The network uses a skip connection between the first fully connected layer and the third fully connected layer so that the output of layer 6 is summed with the output of layer 4 to generate inputs to layer 7. do. Standard batch normalization can be used for all layers except the last layer with a attenuation of 0.97. The use of skip connections and batch normalization can improve the ability to propagate gradients over the network.

The choice of machine learning model can affect the performance of the system. Machine learning model configurations can be divided into two types of considerations. The first is the internal architecture of the model, the choice of model type (convolutional neural network, recurrent neural network, random forest, etc., generalized non-linear regression), as well as the parameters that characterize the implementation of the model (generally the parameters). Means number and / or number of layers, number of decision trees, etc.). The second is the model's external architecture, the placement of the data supplied to the model, and the specific parameters of the problem the model is looking for to solve. The external architecture is partially characterized by the dimensions and types of data provided as input to the model, the time range that the data covers, and the pre- or post-processing that is performed on the data.

Generally speaking, the choice of external architecture is to increase the number of parameters and the amount of information provided as input, which can enhance the predictive power of machine learning models, and to train and evaluate larger models. The balance between the available storage and computing power of the device and the availability of a sufficient amount of data to prevent overfitting.

Many variations of the external architecture of the model discussed in some embodiments are possible. The number of input vectors, as well as the absolute length (number of elements) and time covered, can be changed. Each input vector does not have to be the same length or cover the same period. The data need not be sampled evenly over time and may provide, for example, a 6-hour history of heart rate data, not limited, and data less than 1 hour before t is sampled at a rate of 1 Hz. Data before 1 hour of t and less than 2 hours of t are sampled at a rate of 0.5 Hz, data older than 2 hours are sampled at a rate of 0.1 Hz, and t is the reference time. ..

Figure 5B shows a deployed and trained RNN500. The input data 513 (P _t , R _t , and T _t ) is input to the state (S _t ) 514 at time t and the trained weight 516 is applied. The output of cell (C _t ) 518 is predicted at time t + 1.

520, and updated state S _{t + 1} 522. Similarly, at C _{t + 1} 524, the input data (P _{t + 1} , R _{t + 1} , and T _{t + 1} ) 513'is _{input to St + 1} 522, and the trained weight 516 is applied. The output of C _{t + 1 524 is}

It is 523. As mentioned above, S _{t + 1} is the result of updating _{S t , so S t + 1} has a memory from _{S t from the operation in cell (C t} ) 518 in the previous time step. .. This process continues for n steps, the input data (P _n , R _n , and T _n ) 513'' is _{input to S n} 530 and the trained weight 516 is applied. The output of cell C _{t is predicted 532}

Is. In particular, trained RNNs apply the same weights throughout, but importantly, the state is updated from the previous time step, providing the RNN with the benefits of memory from the previous time step. Those skilled in the art will appreciate that the order of time for entering dependent health indicator data may change and will still produce the desired results. For example, the measured health index data from the previous time step (eg P _t-1 ) and the other factor data from the current time step (eg R _t and T _t ) are the states at the current time step (eg R t and T t). Can be entered in S _t ), where the model is a health indicator at the current time step.

And that health indicator, as described above, is the health indicator data measured at the current time step to determine if the user's health indicator is normal or within the health range. Is compared with.

FIG. 5C shows an alternative embodiment of a trained RNN to determine if a user's health index sequence data, in this example the PPG, is within the band or threshold of a healthy person. The input data in this embodiment is a linear combination.

And here,

Is the predicted health index value at time t, _{and P t is the measured health index value at time t.} In this embodiment, α is non-linearly in the range 0 to 1 as a function of loss (L), where loss and α are discussed in more detail below. It is worth noting here that if α is close to zero, the measured data P _t is input to the network, and if α is close to 1, the predicted data.

Is entered into the network to make predictions in the next time step. Other factor data (O _t ) at time t can also be entered as an option.

I _t and O _t is input to state S _t, state S _t can, in some embodiments, the health index data predicted at time step t + 1

Probability distribution (β),

Is output, where β _{(p *)} is the probability distribution function of the predicted health index (P ^*). In some embodiments, the probability distribution function is the predicted health index value at t + 1.

Is sampled to select. As will be appreciated by those skilled in the art, β _{(p *)} can be sampled using different methods depending on the goals of the network designer, which method is the mean, maximum, or random sampling of the probability distribution. May include taking. ^{Evaluating β t + 1} using the data measured at time t + 1 provides the probability that the state S _{t + 1} would have predicted for the measured data.

To illustrate this concept, FIG. 5D shows a virtual probability distribution for a range of virtual health index data at time t + 1. This function is a predicted health index at time t + 1.

Is sampled, for example, with a maximum probability of 0.95. Probability distribution (β ^{t + 1} ) is also measured or actual health index data

Is evaluated to determine the probability that the model would have predicted if the actual data were entered into the model. In this example,

Is 0.85.

As predicted by a trained machine learning model, losses may be defined to help determine whether to notify the user that the user's health is not within normal limits. Losses are selected to model how close the predicted data is to the actual or measured data. Those of skill in the art will recognize many ways to define losses. In another embodiment described herein, for example, the absolute value (| ΔP ^* |) of the difference between the predicted data and the actual data is the loss. In some embodiments, the loss (L) can be L = -ln [β _(p) ], where

Is. L is a measure of how close the predicted data is to the measured or actual data. β _(p) ranges from 0 to 1, where 1 means that the predicted and measured values are the same. Therefore, a low loss indicates that the predicted value is likely to be the same as or close to the measured value, which in this context means that the measured data appears to be from a healthy / normal person. do. In some embodiments, a threshold for L is set, eg, L> 5, where the user is informed that the health indicator data is outside the range considered healthy. Other embodiments may take the average of the losses over time and compare the average to the threshold. In some embodiments, the threshold itself can be a statistical calculation of the predicted values or a function of the average of the predicted values. In some embodiments, the threshold itself may use the following formula to notify the user that the health indicator is not within the health range,

(P _range ) is determined by the method of averaging the measured health index data over time.

Is determined by a method of averaging predicted health index data over the same time range.

Is the median sequence of standard deviations derived from the network over the same time range.

teeth,

It is a function of the standard deviation evaluated in and can serve as a threshold.

The averaging methods that can be used include, but are not limited to, averaging, arithmetic mean, median, and mode. In some embodiments, outliers are removed so as not to distort the calculated numbers.

Input data for the embodiment shown in FIG. 5C

With reference to, α _t is defined as a function of L and ranges from 0 to 1. For example, α (L) can be a linear or non-linear function, or it can be linear over one range of L and non-linear over another range of L. In one embodiment, as shown in FIG. 5E, the function α (L) is linear with respect to L between 0 and 3, quadratic with respect to L between 3 and 13, and is greater than 13. It is 1 for L. For this embodiment, if L is between 0 and 3 (ie, if the predicted health index data and the measured health index data are in close agreement), then α-1 is close to zero, so input. data I _{t + 1} becomes substantially measurement data _{Pt + 1.} If L is large, for example greater than 13, α (L) is 1, which is the input data.

Is the predicted health index at time t + 1. If L is between 1 and 13, α (L) changes quadratically, and the relative contribution of the predicted and measured health index data to the input data also changes. .. A linear combination of the predicted and measured health indicator data weighted by α (L) is, in this embodiment, to the input data between the predicted and measured data at any particular time step. Allows weighting. In all these examples, the input data may also include _{other factor data (O t).} This is just one example of self-sampling, where any combination of predictive and measurement data is used as input to the trained network. Those of skill in the art will appreciate that many others can be used.

The machine learning model in the embodiment uses a trained machine learning model. In some embodiments, the machine learning model uses a recurrent neural network that requires a trained RNN. As an example, but not as a limitation, FIG. 6 shows an RNN deployed to demonstrate training of RNNs with some embodiments. Cell 602 has an initial state S ₀ 604 and a weight matrix W 606. Step rate data R ₀ , temperature data T ₀ , and initial PPG data P _{0 at} time step zero are input to state S ₀ , a weight W is applied, and the predicted PPG at the first time step.

Is output from cell 602

Is calculated using _{the PPG (P 1} ) obtained in time step 1. _{Cell 602 also outputs the updated state 608 (S 1} ) in time step 1, which state enters cell 610. Step rate data R ₁ , temperature data T ₁ , and PPG data P _{1 in time step 1} are input to state S 1, a weight of 606 W is applied, and the predicted PPG in time step 2.

Is output from cell 610,

Is calculated using _{the PPG (P 2} ) obtained in time step 2. _{Cell 610 also outputs the updated state 612 (S 2} ) in time step 2, which state enters cell 614. The step rate data R ₃ , the temperature data T ₃ , and the PPG data P ₃ in time step 3 are input to S ₂ , the weight 606W is applied, and the predicted PPG in time step 3 is applied.

Is output from cell 614

Is calculated using _{the PPG (P 3} ) obtained in time step 3. This outputs state 616 at time step n,

Continues until is calculated. ΔP ^* 'is used in backpropagation to adjust the weight matrix, similar to training convolutional neural networks. However, unlike convolutional networks, the same weight matrix in the recurrent neural network is applied at each iteration and changes only in backpropagation during training. Many training examples with health index data and corresponding other factor data are entered into the RNN600 multiple times until the RNN600 converges. As discussed earlier, LTSM RNNs can be used in some embodiments where the state of such a network provides a longer-term contextual analysis of the input data, which is that the network is (more) long-term. It can provide better predictions when learning correlations. Also mentioned and, as will be readily appreciated by those of skill in the art, other machine learning models fall within the scope of the embodiments described herein, such as, but not limited to, CNNs or other feedforward networks. May include.

Figure 7A is a system that predicts whether a user's measured health index is within or outside the normal threshold for a healthy person's health index under similar other factors. Shows 700. The system 700 has a machine learning model 702 and a health detector 704. Embodiments with respect to the machine learning model 702 include, for example, (but not limited to) a trained machine learning model, a trained RNN, CNN, or other feedforward network. Trained RNNs, other networks, or combinations of networks can be trained in training examples from a population of healthy individuals for which health indicator data and (temporally) corresponding other factor data have been collected. Alternatively, trained RNNs, other networks, or combinations of networks can be trained in training examples from specific users, resulting in personalized and trained machine learning models. Those skilled in the art will appreciate that training examples from different populations may be selected depending on the application or design of the trained network and system in general. Those skilled in the art will also readily appreciate that the health index data in this embodiment and other embodiments can be one or more health indicators. For example, one or more of PPG data, heart rate data, blood pressure data, body temperature data, blood oxygen concentration data, etc., without limitation, may be used to train the model and predict the health of the user. .. The health detector 704 is used to determine if the loss, or other metric determined by analyzing the predicted output using the measured data, exceeds the threshold considered normal and is therefore unhealthy. , Prediction 708 from machine learning model 702 and input data 710 are used. The system 700 outputs a notification or the health status of the user. This notice may take many forms as discussed herein. The input generator 706 continuously acquires sensor data from the user wearing or in contact with the sensor (not shown), which is one or more health indicators of the user. Represents. Corresponding other factor data (in time) may be collected by another sensor or obtained via other means, as described herein or readily apparent to those of skill in the art. obtain.

The input generator 706 may also collect data to determine / calculate other factor data. The input generator is, for example, a combination of, but not limited to, smartwatches, wearables or mobile devices (eg, Apple Watch® or FitBit® smartphones, tablets, or laptop computers), smartwatches and mobile devices. It may include a surgically implanted device capable of transmitting data to a mobile device or other portable computing device, or a device on a cart in a medical facility. Preferably, the user input generator 706 has a sensor (eg, a PPG sensor, an electrode sensor) for measuring data related to one or more health indicators. Some embodiments of smartwatches, tablets, cell phones, or laptop computers may be equipped with sensors, or the sensors may be placed remotely (either surgically implanted or the body away from the mobile device). In all of these cases, the mobile device communicates with the sensor to collect health indicator data. In some embodiments, the system 700 is alone on a mobile device, in combination with other mobile devices, or in combination with other computing systems via communication over a network through which these devices can communicate. Can be provided. For example, without limitation, the system 700 can be a smartwatch or wearable in which the machine learning model 702 and the health detector 704 are located on the device, eg, in the memory of the wristwatch or in the firmware on the wristwatch. The watch has a user input generator 706, via direct communication, wireless communication (eg, WiFi, sound, Bluetooth®, etc.), or network (eg, internet, intranet, extranet, etc.), or them. A trained machine learning model 702 and health detector 704 that can communicate with other computing devices (eg, mobile phones, tablets, laptop computers, or desktop computers) through a combination of other computing devices. Can be placed on top. Those skilled in the art will appreciate that any number of configurations of the system 700 may be utilized without going beyond the scope of the embodiments described herein.

Referring to FIG. 7B, a smartwatch 712 according to one embodiment is shown. The smartwatch 712 includes a watch 714 that includes all circuits and microprocessors, as well as processing devices (not shown) known to those of skill in the art. Watch 714 also includes user health index data 718, display 716 in which heart rate data may be displayed in this example. Also displayed on display 716 can be the predicted health indicator band 720 for a normal or healthy population. In FIG. 7B, the user's measured heart rate data does not exceed the predicted health band, so no notification will be given in this particular example. The watch 714 may also include a watch band 722 and a high fidelity sensor 724, for example an ECG sensor. Alternatively, the watch band 722 can be an expandable cuff for measuring blood pressure. For example, the low fidelity sensor 726 (shaded) watches 714 to collect user health indicator data such as PPG data, which can be used to derive heart rate data, or other data such as blood pressure. It is provided on the back of the. Alternatively, as will be appreciated by those of skill in the art, in some embodiments, fitness bands such as FitBit or Polar may be used, which have similar processing power and other factor measurement devices (eg, ppg and). It has an accelerometer).

FIG. 8 shows an embodiment of Method 800 for continuously monitoring a user's health status. Step 802 receives user input data, which includes data on one or more health indicators (also known as primary sequence of data) and data on other factors (also known as temporally corresponding). (Secondary sequence of) and can be included. Step 804 inputs user data into a trained machine learning model, where the trained machine learning model is a trained RNN, CNN, other feedforward network as described herein, or the present. It may include other neural networks known to the vendor. In some embodiments, the health indicator input data is one or a combination of the predicted health indicator data and the measured health indicator data, as described in some embodiments herein. , For example, it can be a linear connection. Step 806 outputs data about one or more predicted health indicators in a time step, the output of which, by way of example, includes, but is not limited to, a single predicted value, a probability distribution as a function of the predicted values. obtain. Step 808 determines the loss based on the predicted health index, where, for example, but not limited, the loss is a simple difference between the predicted health index and the measured health index, or what. Or other well-selected loss function (eg, the negative logarithm of the probability distribution evaluated in the value for the measured health index). Step 810 determines if the loss exceeds a threshold that is considered normal or unhealthy, where the threshold is, for example, not a limitation, but a simple number selected by the designer, or a prediction. It can be a more complex function of some related parameter. If greater than the threshold, step 812 notifies the user that the user's health index exceeds the threshold considered normal or healthy. As described herein, notifications can take many forms. In some embodiments, this information can be visualized to the user. For example, without limitation, the information was generated by (i) measured health index data (eg, heart rate) and other factor data (eg, steps) as a function of time, and (ii) a machine learning model. It may be displayed on a user interface such as a graph showing the distribution of predicted health indicator data (eg, predicted heart rate values). In this way, the user visually compares the measured data points with the predicted data points to see if the user's heart rate is, for example, within the range expected by the machine learning model. It can be judged by a target inspection.

Some embodiments described herein refer to the use of thresholds to determine whether to notify the user. In one or more of these embodiments, the user may change the threshold to adjust or tune the system or method to more closely match the user's personal health knowledge. For example, if the physiological indicator used is blood pressure and the user has a higher blood pressure, the embodiment indicates that the user's indicator is outside the normal or healthy range from a trained model in a healthy population. May frequently alert / notify users. Thus, certain embodiments allow the user to raise the threshold so that the user is less often notified that the user's health index exceeds what is considered normal or healthy.

Some embodiments preferably use raw data on health indicators. If the raw day is processed to derive a particular measurement, eg heart rate, this derived data can be used according to embodiments. In some situations, the provider of the health monitoring device has no control over the raw data and what is received is the calculated health index, eg, the data processed in the form of heart rate or blood pressure. As will be understood by those skilled in the art, the form of the data used to train the machine learning model should match the form of the data collected from the user and entered into the trained model, otherwise. If so, the prediction may turn out to be incorrect. For example, the Apple Watch provides heart rate measurement data in unequal time steps, not raw PPG data. In this example, the user wears an Apple Watch that outputs heart rate data according to Apple's PPG processing algorithm using heart rate data at uneven time steps. The model is trained on this data. Apple has decided to modify its algorithm to provide heart rate data so that the model trained in the data from the previous algorithm will not be used in the data input from the new algorithm. there's a possibility that. To account for this potential problem, some embodiments resample irregularly spaced data (such as heart rate, blood pressure data, or ECG data) into a regularly spaced grid and model. When collecting data for training, sample from a regularly spaced grid. If Apple, or any other provider of data, modifies the algorithm, the model only needs to be retrained in the newly collected training examples, and the model will be rebuilt to account for the algorithm changes. There is no need to.

In a further embodiment, the trained machine learning model can be trained on the user's data, resulting in a personalized and trained machine learning model. This trained personalized machine learning model may be used in place of or in combination with the trained machine learning model in the population of healthy people described herein. When used alone, the user's data is populated into a personalized and trained machine learning model, which is normal for that user in the next time step. Outputs a prediction of a health index, which is then used herein to determine if a user's health index differs from what is expected to be normal for that user by some threshold. It is compared with the actual / measured data from the next time step in a manner consistent with the embodiment described in. In addition, this personalized machine learning model has predictions and related notifications related to both what was predicted to be normal for its individual users and what was predicted to be normal for a population of healthy people. Can be used in combination with a trained machine learning model in training examples from a population of healthy people to generate and.

FIG. 9A shows method 900 according to another embodiment, and FIG. 9B shows a virtual plot 902 of heart rate (eg, but not limited) as a function of time for purposes of illustration. Step 904 (Figure 9A) receives the user's heart rate data (or other health indicator data) and optionally the corresponding (temporally) other factor data and trains this data personalized. Enter into the machine learning model. In some embodiments, the personalized and trained model is trained on the user's individual health indicator data and, optionally, the corresponding other data (in time) as described herein. Will be done. Therefore, in step 906, a personalized and trained machine learning model predicts normal heart rate data for that individual user under the condition of other factors, and step 908 for that particular user. Identify deviations or abnormalities in the user's health index data as compared to what was predicted to be normal. Some embodiments allow the user from a wearable device at the user (eg, Apple Watch, smartwatch, FitBit®, etc.) or from another mobile device (eg, tablet, computer, etc.) that communicates with the sensor at the user. Health indicator data is received and will be discussed throughout this specification.

Losses may be defined to help determine in step 908 whether to notify the user that the user's measurement data is abnormal with respect to what is expected to be normal for that particular user. Losses are selected to model how close the forecast is to the actual or measured data. Those of skill in the art will recognize many ways to define losses. In other embodiments described herein and again equally applicable, for example, the absolute value | ΔP ^* | of the difference between the predicted and absolute values is the form of loss. In some embodiments, the loss (L) can be L = ln [β _(p) ], where

Is. L is generally
It is a measure of how close the forecast data is to the measured data. β _(p) , in this example the probability distribution ranges from 0 to 1, where 1 means that the predicted and measured data are the same. Therefore, a low loss indicates that, in some embodiments, the predicted data is likely to be the same as or close to the measured data. In some embodiments, a threshold for L is set, eg, L> 5, where the user is informed that an abnormal state exists from the predicted state for that particular user. To. This notice may take many forms, as described elsewhere herein. As described elsewhere herein, other embodiments may take an average of losses over time and compare that average to a threshold. In some embodiments, the threshold itself can be a statistical calculation of the forecast data or a function of the average of the forecast data, as described in more detail elsewhere herein. Losses are described in more detail elsewhere herein and are not discussed further here for brevity. Those skilled in the art will also appreciate that input and forecast data can be scalar values, or segments of data over time. For example, but not limited, a system designer may be interested in a 5 minute data segment, entering all data before time t and all other data for t + 5 minutes. Predict the t + 5 minute health indicator data and determine the loss between the measured health indicator data for the t + 5 minute segment and the predicted health indicator data for the t + 5 minute segment.

Step 908 determines if an anomaly is present. As discussed, this can be determined if the loss exceeds the threshold. As mentioned above, the thresholds are set by the designer's choice based on the purpose of the system being designed. In some embodiments, the threshold can be changed by the user, but in this embodiment it is not preferred. If no anomalies are present, the process is repeated in step 904. If anomalies are present, step 910 notifies or warns the user to obtain a high fidelity measurement, eg, an ECG or blood pressure measurement, but not a limitation. In step 912, the high fidelity data is analyzed by algorithms, medical professionals, or both and described as normal or abnormal, and if not, some diagnosis, depending on the high fidelity measurements obtained. For example, AFib, tachycardia, bradycardia, atrial flutter, or high / hypotension may be assigned. For clarity, the notice for recording high fidelity data is equally applicable and possible in other embodiments and in certain embodiments using the general model described above. Please note that. High fidelity measurements may be obtained directly by the user in some embodiments using a mobile surveillance system such as an ECG or blood pressure system, which in some embodiments is associated with a wearable device. Can be. Alternatively, notification step 910 triggers the automatic acquisition of high fidelity measurements. For example, a wearable device may communicate with a sensor (via a wired or wireless connection) and obtain ECG data, or a wearable device may have a blood pressure cuff system (eg, wearable) to automatically obtain blood pressure measurements. Wristbands, or armband cuffs), or wearable devices can communicate with embedded devices such as pacemakers or ECG electrodes. Systems for remote acquisition of ECG are provided, for example, by AliveCor, such systems including (but not limited to) one or more sensors that contact the user in more than one place. Here, the sensor collects electrocardiographic data, which is sent to the mobile computing device wired or wirelessly, the app generates an ECG strip from the data, and the ECG strip is an algorithm, medical It can be analyzed by an expert, or both. Alternatively, the sensor can be a blood pressure monitor and blood pressure data is transmitted to the mobile computing device either wired or wirelessly. The wearable itself can be a blood pressure system with a cuff capable of measuring health index data and optionally has an ECG sensor similar to that described above. ECG sensors may also include ECG sensors such as those described in the shared US Provisional Application No. 61 / 872,555, the contents of which are incorporated herein by reference. Mobile computing devices are, for example, not limited to computer tablets (eg, iPad®), smartphones (eg, iPhone®), wearables (eg, Apple Watch), or in medical facilities (probably). It can be a device (on the cart). The mobile computing device can, in some embodiments, be a laptop computer or computer that communicates with some other mobile device. Those skilled in the art will appreciate that wearables or smartwatches are also considered mobile computing devices in terms of the functionality provided in the context of the embodiments described herein. In the case of a wearable, the sensor can be placed on a band of wearables, where the sensor can send data to the computing device / wearable wirelessly or by wire, or the band can also be a blood pressure monitoring cuff, or as described above. Can be both. For mobile phones, the sensor can be a pad attached to the phone, or a pad away from the phone, which senses electrical heart signals and wears that data wirelessly or by wire or other mobile computing. Communicate with the ing device. A more detailed description of some of these systems is one of U.S. Pat. Nos. 9,420,956, 9,572,499, 9,351,654, 9,247,911, 9,254,095, and 8,509,882. Provided in plural and in one or more of U.S. Patent Application Publication Nos. 2015/0018660, 2015/0297134, and 2015/0320328, all of which are all in their entirety. Incorporated herein for purposes. Step 912 analyzes the high fidelity data and provides an explanation or diagnosis, as described above.

In step 914, the diagnosis or classification of high fidelity measurements is received by the computing system, which, in some embodiments, collects the user's heart rate data (or other health indicator data). It can be a mobile or wearable computing system used for, in step 916, a low fidelity health indicator data sequence (in this example, heart rate data) is labeled diagnostically. In step 918, the labeled user's low fidelity data sequence is used to train the high fidelity machine learning model, and optionally other factor data sequences are also provided to train the model. Trained high-fidelity machine learning models, in some embodiments, receive measured low-fidelity health indicator data sequences (eg, heart rate data or PPG data) and optionally other factor data. It provides probabilities or predictions, or has the ability to diagnose or detect when a user is experiencing an event that is typically diagnosed or detected using high fidelity data. Trained high-fidelity machine learning models can do this because they are trained in the user's health index data (and optionally other factor data) labeled with high-fidelity data diagnostics. Therefore, the trained model is based solely on the measured low fidelity health indicator input data sequence, eg heart rate or ppg data (and optionally other factor data), and the user labels (eg, Afib, hypertension, etc.). ) Has the ability to predict if it has an event associated with one or more of them. Those skilled in the art will appreciate that training of high fidelity models can be performed on the user's mobile device, away from the user's mobile device, in combination of the two, or in a distributed network. For example, but not limited to, user health indicator data may be stored in the cloud system and this data may be labeled in the cloud using the diagnostics from step 914. One of ordinary skill in the art will readily understand any number of methods and methods for storing, labeling and accessing this information. Alternatively, a globally trained high fidelity model can be used, and this globally trained high fidelity model is typically diagnosed or detected using high fidelity measurements. Trained in labeled training examples from a group of people experiencing the condition. These global training examples are low fidelity data sequences labeled with high fidelity measurements (eg, Afib called from the ECG by a healthcare professional or algorithm) and labeled with a low fidelity data sequence (eg, heart rate). Number) is provided.

Referring here to FIG. 9B, plot 902 shows a schematic diagram of heart rate plotted as a function of time. Abnormal 920 from the user's normal heart rate data occurred at _{hours t 1} , t ₂ , t ₃ , t ₄ , t ₅ , t ₆ , t ₇ , and t _8. Normal means that the prediction data for this particular user was within the threshold of the measurement data, as described above, and the anomaly is outside the threshold. In anomalies from normal, some embodiments have clearer or higher fidelity readings, eg, but not limited to ECG ₁ , ECG ₂ , ECG ₃ , ECG ₄ , ECG ₅ , ECG ₆ , ECG. _7. Prompt the user to get an ECG reading identified as _{ECG 8.} As described above, the high fidelity reading can be obtained automatically and the user can obtain it, which can be something other than ECG, eg blood pressure. High fidelity readings are analyzed by algorithms, medical professionals, or both to identify high fidelity data as normal / anomalous and to identify / diagnose anomalies, eg, AFib, not limited. Will be done. This information is used to label health index data (eg, heart rate or PPG data) at 920 anomalies in the user's sequence data.

The distinction between high fidelity data and low fidelity data is that high fidelity data or measurements are typically used to make decisions, detections, or diagnoses, such as low fidelity data. It is not easy to use. For example, ECG scans can be used to identify, detect, or diagnose arrhythmias, but heart rate or PPG data typically does not provide this function. Those skilled in the art will appreciate the description of this specification relating to machine learning algorithms (eg, Bayesian, Markov, Gaussian processes, clustering algorithms, generative models, kernels, and neural network algorithms) in all embodiments described herein. You will understand that it applies equally.

In some situations, the user remains asymptomatic despite the possibility of problems, and the high fidelity measurements required to make a diagnosis or detection, even in the presence of symptoms. Getting the value can be unrealistic. For example, not limited, arrhythmias, especially AF, may not appear, and even if symptoms do appear, recording ECG at that moment is more difficult, expensive, and bulky, as it is notorious. It is very difficult to continuously monitor a user without a monitoring device, which is sometimes invasive. As discussed elsewhere in this specification, AF can contribute to stroke, at least in other serious conditions, so it is not understandable when a user experiences AF. ,is important. Similarly, as discussed elsewhere, AF burdens can be of similar importance. Some embodiments are continuous monitoring of arrhythmias (eg AF) or other serious conditions using only continuous monitoring of low fidelity health indicator data such as heart rate or ppg and optional other factor data. Enables monitoring.

FIG. 10 shows method 1000 according to some embodiments of health monitoring systems and methods. Step 1002 includes measured or actual user low fidelity health index data (eg, heart rate from a sensor on the wearable or PPG) and optionally health index data as described herein. Receive the corresponding (temporal) other factor data that may affect it. As discussed elsewhere herein, low fidelity health indicator data can be measured by mobile computing devices such as smartwatches, other wearables, or computer tablets. In step 1004, the user's low fidelity health index data (and optionally other factor data) is input to the trained high fidelity machine learning model, and the trained high fidelity machine learning model is in step 1006. Outputs the predicted identification or diagnosis for the user based on the measured low fidelity health index data (and optionally the corresponding (temporally) other factor data). Step 1008 asks if the identification or diagnosis is normal, and if so, the process is redone. If the identification or diagnosis is not successful, step 1010 informs the user of the problem or detection. Optionally, the system, method, and platform can be configured to notify any combination of users, family, friends, medical professionals, emergency 911, and so on. Who of these persons is notified may depend on identification, detection, or diagnosis. If identification, detection, or diagnosis is life-threatening, you may not be notified if the diagnosis is not life-threatening. You may be contacted or notified to a particular person. In addition, in some embodiments, the measured health indicator data sequence is input to a trained high fidelity machine learning model and the amount of time the user is experiencing anomalous events (eg, predicted). The difference between the start and stop of anomalous events) is calculated, allowing a better understanding of the anomalous burden of the user. Understanding AF burden can be very important, especially in preventing stroke and other serious conditions. Therefore, some embodiments are for anomalous events using low fidelity health factor data and optionally mobile computing devices, wearable computing devices, or other portable devices that can only acquire other factor data. Enables continuous monitoring.

FIG. 11 shows exemplary data 1100 analyzed on the basis of low fidelity data to generate high fidelity output predictions or detections according to some of the embodiments described herein. Although described with reference to the detection of atrial fibrillation, similar data may be generated for additional prediction of high fidelity diagnosis based on low fidelity measurements. The first chart 1110 shows the heart rate calculation over time for the user. Heart rate can be determined based on PPG data or other heart rate sensors. The second chart 1120 shows activity data for users during the same time period. For example, activity data can be determined based on steps, or other measurements of user movement. The third chart 1130 shows the classifier output from the machine learning model and the horizontal threshold when the notification is generated. Machine learning models can generate predictions based on the input of low fidelity measurements. For example, the data in the first chart 1110 and the second chart 1120 can be analyzed by a machine learning system as further described above. The results of the machine learning system analysis can be provided as the probability of atrial fibrillation shown in Chart 1130. If the probability exceeds the threshold, which is shown as a confidence level greater than 0.6 in this case, the health monitoring system triggers a notification or other alert to the user, doctor, or other user associated with the user. can do.

In some embodiments, the data in charts 1110 and 1120 may be provided to the machine learning system as continuous measurements. For example, heart rate and activity levels can be generated as measurements every 5 seconds for accurate measurements. A time segment with multiple measurements can then be entered into the machine learning model. For example, the data from the previous hour can be used as input to a machine learning model. In some embodiments, shorter or longer time periods may be provided instead of one hour. As shown in FIG. 11, output chart 1130 provides an indicator of the time period during which a user is experiencing an abnormal health event. For example, periods when predictions exceed a certain level of confidence can be used by health monitoring systems to determine atrial fibrillation. This value can then be used to determine the atrial fibrillation burden in the user during the measured time period.

In some embodiments, the machine learning model for generating the predicted output in Chart 1130 can be trained on the basis of labeled user data. For example, labeled user data is high fidelity data (ECG) taken over a period of time when low fidelity data (eg, PPG, heart rate) and other data (eg, activity level or steps) are also available. Can be provided based on readings, etc.). In some embodiments, the machine learning model is designed to determine if atrial fibrillation is likely to have been present during the previous time period. For example, a machine learning model can take 1 hour of low fidelity data as input and provide the likelihood that an event was present. Therefore, training data may include several hours of recorded data about a population of individuals. The data can be health event labeled times when the condition is diagnosed based on high fidelity data. Therefore, if there is a health event labeled time based on high fidelity data, the machine learning model will have any one hour window of low fidelity data with that event entered into the untrained machine learning model. It can be determined that predictions of health events should be provided. The untrained machine learning model can then be updated based on comparing the predictions with the labels. After repeating several iterations and determining that the machine learning model has converged, the machine learning model can be used by a health monitoring system to monitor the user's atrial fibrillation based on low fidelity data. In various embodiments, other conditions than atrial fibrillation can be detected using low fidelity data.

FIG. 12 is a schematic representation of a machine in an exemplary form of computer system 1200, in which one or more of the methodologies discussed herein are machined. A set of instructions to be executed can be executed. In an alternative embodiment, the machine may be connected (eg, networked) to a local area network (LAN), an intranet, an extranet, or another machine in the Internet. The machine can operate as a server or client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machines include personal computers (PCs), tablet PCs, set-top boxes (STBs), personal digital assistants (PDAs), mobile phones, web appliances, servers, network routers, switches or bridges, hubs, access points, network access control devices. , Or any machine capable of executing a set of instructions (sequential or otherwise) that specifies the action to be taken by that machine. Further, although only a single machine is exemplified, the term "machine" also refers to a set of instructions (or to execute any one or more of the methodologies discussed herein. It should be construed to include any collection of machines that run multiple sets) individually or jointly. In one embodiment, the computer system 1200 may be typical of a server, mobile computing device, wearable, etc. configured to perform health monitoring as described herein.

An exemplary computer system 1200 has a processing device 1202, main memory 1204 (eg, read-only memory (ROM), flash memory, dynamic random access memory (DRAM)), and static memory that communicate with each other via bus 1230. Includes 1206 (eg, flash memory, static random access memory (SRAM), etc.) and data storage device 1218. Any of the signals provided via the various buses described herein may be time-division-multiplexed with other signals and provided via one or more common buses. In addition, interconnections between circuit components or blocks may be shown as a bus or a single signal line. Each of the buses may be an alternative single signal line, and each of the single signal lines may be an alternative bus.

Processing device 1202 represents one or more general purpose processing devices, such as a microprocessor, central processing unit, or other processing device. More specifically, the processing device implements a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or other instruction set. It can be a processor, or a processor that implements a combination of instruction sets. The processing device 1202 can also be one or more dedicated processing devices such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, and the like. Processing device 1202 is configured to perform processing logic 1226, which may be an example of a health monitor 1250 and related systems for performing the actions and steps discussed herein.

The data storage device 1218 may include a machine-readable storage medium 1228, on which the machine-readable storage medium 1228 is instructed to cause the processing device 1202 to perform the health monitor 1250 and related processes as described herein. One or more sets of instructions 1222 (eg, software) embody any one or more of the functional methodologies described herein are stored, including. Instruction 1222 may be present entirely or at least partially in main memory 1204 or in processing device 1202 during its execution by computer system 1200, and main memory 1204 and processing device 1202 also configure a machine-readable storage medium. do. Instruction 1222 may be further transmitted or received over network 1220 via network interface device 1208.

Machine-readable storage medium 1228 may also be used to store instructions for performing methods for monitoring user health, as described herein. Machine-readable storage medium 1228 is shown in an exemplary embodiment as being a single medium, but the term "machine-readable storage medium" is used to store one or more sets of instructions. It should be interpreted to include one medium or multiple media (eg, centralized or distributed databases, or associated caches and servers). Machine-readable media include any mechanism for storing information in a form readable by a machine (eg, a computer) (eg, software, processing application). Machine-readable media are, but are not limited to, magnetic storage media (eg, floppy (registered trademark) diskettes), optical storage media (eg, CD-ROM), optomagnetic storage media, read-only memory (ROM), random access memory. It may include (RAM), erasable programmable memory (eg EPROM and EEPROM), flash memory, or another type of medium suitable for storing electronic instructions.

The aforementioned description describes a number of specific details, such as examples of specific systems, components, methods, etc., in order to provide a better understanding of some embodiments of the present disclosure. However, it will be apparent to those skilled in the art that at least some embodiments of the present disclosure may be practiced without these particular details. In other examples, well-known components or methods are not described in detail or are presented in simple block diagram format in order to avoid unnecessarily obscuring the disclosure. Therefore, the specific details given are merely exemplary. Specific embodiments may differ from these exemplary details and are still considered to be within the scope of the present disclosure.

In addition, some embodiments may be implemented in a distributed computing environment in which machine-readable media are stored on or more than two computer systems and / or run by more than one computer system. In addition, the information transferred between the computer systems can be pulled or pushed through the communication medium connecting the computer systems.

Embodiments of the claimed subject matter include, but are not limited to, the various actions described herein. These operations may be performed by hardware components, software, firmware, or a combination thereof.

The actions of the methods herein are shown and described in a particular order, but the order of actions of each method is such that certain actions may be performed in reverse order, or certain actions may be performed. It can be modified to be performed at the same time as other actions, at least in part. In another embodiment, the instructions or sub-actions of the separate actions can be intermittent or alternating.

The above description of the exemplary implementations of the invention, including those described in the abstract, is not intended to be exhaustive and to limit the invention to the exact embodiments disclosed. No. Specific embodiments of the invention and examples thereof are described herein for illustrative purposes, but as will be appreciated by those skilled in the art, various equivalent modifications are possible within the scope of the invention. .. The word "example" or "exemplary" is used herein to mean serve as an example, an example, or an example. Any aspect or design described herein as an "example" or "exemplification" should not necessarily be construed as preferred or advantageous over other aspects or designs. Rather, the use of the word "example" or "exemplification" is intended to present the concept concretely. As used in this application, the term "or" is intended to mean a comprehensive "or" rather than an exclusive "or". That is, unless otherwise stated or otherwise clear from the context, "X contains A or B" is intended to mean any of the natural, inclusive order. That is, if X contains A, X contains B, or X contains both A and B, then "X contains A or B" is satisfied under any of the above examples. .. In addition, the articles "a" and "an" used in this application and the appended claims are generally used unless otherwise specified or the context makes it clear that they are singular. It should be interpreted to mean "one or more". Moreover, the use of the terms "embodiment" or "one embodiment" or "implementation" or "one implementation" throughout is intended to mean the same embodiment or implementation unless so described. I haven't. In addition, terms such as "first," "second," "third," and "fourth" as used herein are intended as labels to distinguish different elements and are not necessarily. , May not have an ordered meaning according to those numerical specifications.

Variants of other features and functions disclosed above, or alternatives thereof, can be combined with many other different systems or applications. Various currently unforeseen or unexpected substitutions, modifications, modifications, or improvements within it may be made later by one of ordinary skill in the art, which are covered by the claims below. Is also intended. The claims may include embodiments in hardware, software, or combinations thereof. In addition to the embodiments described above, the present disclosure includes, but is not limited to, the following exemplary implementations.

Several exemplary implementations provide a way to monitor a user's heart health. The method is a step of receiving the measured health index data and the other factor data of the user in the first time, and a step of inputting the health index data and the other factor data into the machine learning model by the processing device. The machine learning model produces predicted health indicator data in the next time step, the step in which the user's data is received in the next time step, and the processing device determines the loss in the next time step. The step and the loss are the steps in which the loss is the measurement between the predicted health indicator data in the next time step and the user's measured health indicator data in the next time step. It can include a step of determining if the value is exceeded and a step of outputting a notification to the user in response to determining that the loss has exceeded the threshold.

In some exemplary implementations of any exemplary implementation method, the trained machine learning model is a trained generated neural network. In some exemplary implementations of any exemplary implementation method, the trained machine learning model is a feedforward network. In some exemplary implementations of any exemplary implementation method, the trained machine learning model is an RNN. In some exemplary implementations of any exemplary implementation method, the trained machine learning model is a CNN.

In some exemplary implementations of any exemplary implementation method, the trained machine learning model is a training example from one or more of a healthy population, a heart disease population, and a user. Have been trained in.

In some exemplary implementations of any exemplary implementation method, the loss at the next time step was measured by the predicted health indicator data at the next time step and the user at the next time step. It is the absolute value of the difference from the health index.

In some exemplary implementations of any exemplary implementation method, the predicted health indicator data is a probability distribution and the predicted health indicator data in the next time step is sampled from the probability distribution. Will be done.

In some exemplary implementations of any exemplary implementation method, the predicted health indicator data at the next time step was predicted from the predicted health indicator data at maximum probability, and the probability distribution. It is sampled according to a sampling technique selected from a group consisting of randomly sampling health index data.

In some exemplary implementations of any exemplary implementation method, the predicted health indicator data is a probability distribution (β) and the loss is the user's measured health in the next time step. It is determined based on the negative logarithm of the probability distribution in the next time step evaluated using the index. In some exemplary implementations of any exemplary implementation, the method further comprises self-sampling of the probability distribution.

In some exemplary implementations of any exemplary implementation of the method, the method is measured by the user over a period of time steps, with a step of averaging predicted health indicator data over a period of time steps. It further includes a step of averaging the health index data and a step of determining the loss based on the difference in absolute value between the predicted health index data and the measured health index data.

In some exemplary implementations of any exemplary implementation method, the measured health indicator data includes PPG data. In some exemplary implementations of any exemplary implementation method, the measured health index data includes heart rate data.

In some exemplary implementations of any exemplary implementation, the method further comprises the step of resampling irregularly spaced heart rate data into a regularly spaced grid, and the heart rate data. Is sampled from a grid with regular intervals.

In some exemplary implementations of any exemplary implementation method, the measured health index data consists of a group consisting of PPG data, heart rate data, pulse oximeter data, ECG data, and blood pressure data. One or more selected health index data.

Some exemplary limitations provide a device comprising a processing device, a display, a health indicator data sensor, and a mobile computing device with memory having stored instructions, the instructions being executed by the processing device. Then, the operation of receiving the health index data measured from the health index data sensor at the time and the other factor data at the first time, and the health index data and the other factor data are trained in the machine learning model. Receives the motion and the measured health indicator data and other factor data in the next time step, which is the motion to be input and the trained machine learning model produces the predicted health indicator data in the next time step. The action to be performed and the action to determine the loss in the next time step, where the loss is the measured value between the predicted health indicator data in the next time step and the measured health indicator data in the next time step. The processing device is made to perform the operation and the operation of outputting a notification when the loss in the next time step exceeds the threshold value.

In some exemplary implementations of any exemplary device, the trained machine learning model comprises a trained generated neural network. In some exemplary implementations of any exemplary device, the trained machine learning model includes a feedforward network. In some exemplary implementations of any exemplary device, the trained machine learning model is an RNN. In some exemplary implementations of any exemplary implementation method, the trained machine learning model is a CNN.

In some exemplary implementations of any exemplary device, the trained machine learning model is trained in training examples from one of a group consisting of a healthy population, a heart disease population, and a user. There is.

In some exemplary implementations of any exemplary device, the predicted health index data is a point prediction of the user's health index at the next time step and the predicted health index at the next time step. The absolute value of the difference between the data and the measured health index data.

In some exemplary implementations of any exemplary device, the predicted health index data is sampled from a probability distribution generated from a machine learning model.

In some exemplary implementations of any exemplary device, the predicted health index data is sampled according to a sampling technique selected from a group consisting of maximum probabilities and random sampling from a probability distribution.

In some exemplary implementations of any exemplary device, the predicted health index data is a probability distribution (β) and the loss uses the user's measured health index in the next time step. Determined based on the negative logarithm of β evaluated in.

In some exemplary implementations of any exemplary device, the processing device further defines a function α in the range 0 to 1 and was predicted to be the user's measured health index data. Construct a linear connection with health index data as a function of α.

In some exemplary implementations of any exemplary device, the processing device further performs self-sampling of the probability distribution.

In some exemplary implementations of any exemplary device, the processing device also uses an averaging method to capture predicted health indicator data sampled from a probability distribution over a period of time steps. Using the averaging and averaging method, the user's measured health metric data is averaged over a period of time steps, between the averaged predicted health metric data and the measured health metric data. It defines the loss based on the difference in absolute values.

In some exemplary implementations of any exemplary device, the averaging method is to calculate the mean, calculate the arithmetic mean, calculate the median, and calculate the mode. Includes one or more methods selected from a group consisting of.

In some exemplary implementations of any exemplary device, the measured health indicator data includes PPG data from the PPG signal. In some exemplary implementations of any exemplary device, the measured health index data is heart rate data. In some exemplary implementations of any exemplary device, heart rate data is collected by resampling irregularly spaced heart rate data into a regularly spaced grid, and the heart rate data is , Sampled from a regularly spaced grid. In some exemplary implementations of any exemplary device, the measured health index data was selected from a group consisting of PPG data, heart rate data, pulse oximeter data, ECG data, and blood pressure data. One or more health index data.

In some exemplary implementations of any exemplary device, the mobile device is selected from a group consisting of smart watches, fitness bands, computer tablets, and laptop computers.

In some exemplary implementations of any exemplary device, the mobile device further comprises a user high fidelity sensor, the notification requires the user to obtain high fidelity measurement data, and the processing device. In addition, receive an analysis of high fidelity measurement data, label the user's measured health indicator data in the analysis to generate labeled user health indicator data, and trained personalized high. Labeled user health index data are used as training examples to train fidelity machine learning models.

In some exemplary implementations of any exemplary device, the trained machine learning model is stored in memory. In some exemplary implementations of any exemplary device, the trained machine learning model is stored on remote memory, the remote memory is separated from the computing device, and the mobile computing device is wearable. It is a computing device. In some exemplary implementations of any exemplary device, the trained personalized high fidelity machine learning model is stored in memory. In some exemplary implementations of any exemplary device, a trained personalized high fidelity machine learning model is stored on remote memory, which is separated from the computing device and mobile. The computing device is a wearable computing device.

In some exemplary implementations of any exemplary device, the processing device further predicts that the user is experiencing atrial fibrillation and determines the user's atrial fibrillation burden. ..

Several exemplary implementations provide a way to monitor a user's heart health. The method personalized the step of receiving the measured low fidelity user health index data and the other factor data in the first time, and the data including the user health index data and the other factor data in the first time. Steps to enter into a high fidelity trained machine learning model, where a personalized, high fidelity trained machine learning model predicts whether the user's health indicator data is abnormal. , If the prediction is abnormal, it may include a step of sending a notification that the user's health is abnormal.

In some exemplary implementations of any exemplary implementation method, a trained personalized high fidelity machine learning model is a measured low labeled in the analysis of high fidelity measurement data. Fidelity trained in user health index data.

In some exemplary implementations of any exemplary implementation method, the analysis of high fidelity measurement data is based on user-specific high fidelity measurement data.

In some exemplary implementations of any exemplary implementation method, the personalized high fidelity machine learning model outputs a probability distribution and predictions are sampled from the probability distribution.

In some exemplary implementations of any exemplary implementation method, predictions are sampled according to a sampling technique selected from a group consisting of predictions at maximum probability and random sampling of predictions from a probability distribution. Random.

In some exemplary implementations of any exemplary implementation method, the averaged predictions are determined by averaging the predictions over a period of time steps using the averaging method. The averaged predictions are used to determine if the user's health index data is normal or abnormal.

In some exemplary implementations of any exemplary implementation method, the averaging method is to calculate the mean, calculate the arithmetic mean, calculate the median, and determine the mode. Includes one or more methods selected from a group consisting of calculations.

In some exemplary implementations of any exemplary implementation method, a personalized, high-fidelity trained machine learning model is stored in the memory of the user's wearable device. In some exemplary implementations of any exemplary implementation method, the measured health index data and other factor data are time segments of the data over time.

In some exemplary implementations of any exemplary implementation method, a personalized, high-fidelity trained machine learning model is stored in remote memory, which is the user's wearable compute. It is located remotely from the training device.

In some exemplary implementations, the health monitoring device may include a mobile computing device comprising a microprocessor, a display, a user health indicator data sensor, and a memory having stored instructions. When executed by the microprocessor, it is an operation to receive the measured low fidelity health index data and the other factor data in the first time, and the measured health index data is acquired by the user health index data sensor. The motion and the data including the health index data and the other factor data in the first time are input to the trained high fidelity machine learning model, and the trained high fidelity machine learning model is , Predicting whether the user's health index data is normal or abnormal, and the action of sending a notification to the user that the user's health is abnormal in response to the prediction being abnormal. To the processing device.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the trained high fidelity machine learning model is a trained high fidelity generation neural network. In some exemplary implementations of any exemplary implementation of the health monitoring device, the trained high fidelity machine learning model is a trained recurrent neural network (RNN). In some exemplary implementations of any exemplary implementation of the health monitoring device, the trained high fidelity machine learning model is a trained feedforward neural network. In some exemplary implementations of any exemplary implementation of the health monitoring device, the trained high fidelity machine learning model is the CNN.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the trained high fidelity machine learning model is measured labeled based on user-specific high fidelity measurement data. Trained in user health index data.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the trained high fidelity machine learning model is a low fidelity health index labeled based on high fidelity measurement data. Trained in the data, low fidelity health index data and high fidelity measurement data are from a population of subjects.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the high fidelity machine learning model outputs a probability distribution and the predictions are sampled from the probability distribution.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the prediction follows a sampling technique selected from a group consisting of predictions at maximum probability and random sampling of predictions from a probability distribution. It will be sampled.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the averaged predictions are determined by averaging the predictions over a period of time steps using an averaging method. The averaged predictions are used to determine if the user's health index data is normal or abnormal.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the measured health index data and other factor data are time segments of the data over time.

In some exemplary implementations of any exemplary implementation of the health monitoring device, the averaging method calculates the average, calculates the arithmetic mean, calculates the median, and calculates the mode. Includes one or more methods selected from a group consisting of.

In some exemplary implementations of any exemplary implementation of the health monitoring device, a personalized, high fidelity trained machine learning model is stored in memory. In some exemplary implementations of any exemplary implementation of the health monitoring device, a personalized, high-fidelity trained machine learning model is stored in remote memory, and the remote memory is wearable compute. It is located remotely from the training device. In some exemplary implementations of any exemplary implementation of the health monitoring device, the mobile device is selected from a group consisting of smart watches, fitness bands, computer tablets, and laptop computers.

402 Expected or likely output data
404 Solid line
500 Trained Recurrent Neural Networks, Trained Neural Networks, RNNs, Deployed Trained RNNs
502 status
504 Predicted PPG
506 steps
508 steps
510 steps
512 steps
513 Input data
513'Input data
513'' Input data
State at 514 hours t
516 Trained weight
518 cell (C _t )
Forecast at 520 hours t + 1
522 Updated state
523 P ^* _{t + 2}
524 C _{t + 1}
530 S _n
532 Forecast
600 RNN
602 cell
604 Initial state S ₀
606 Weight matrix W, weight
608 Time Updated state in step 1
610 cell
Updated state in 612 hour step 2
Cell 614
616 State at time step n
700 system
702 Machine learning model, trained machine learning model
704 Health detector
706 Input generator, user input generator
708 Forecast
710 Input data
712 smart watch
714 watch
716 display
718 User health index data
720 Health Index Band
722 watch band
724 High fidelity sensor
726 Low fidelity sensor
900 methods
902 Virtual plot, plot
920 anomaly
1100 data
1110 1st chart, chart
1120 2nd chart, chart
1130 3rd chart, chart, output chart
1200 computer system, computing system
1202 processing device
1204 main memory
1206 static memory
1208 network interface device
1218 Data storage device
1220 network
1222 instruction
1226 Processing logic
1228 Machine-readable storage medium
1230 bus
1250 health monitor

Claims (20)

With the processing device
With a photopretismography (“PPG”) sensor operably coupled to the processing device,
With a motion sensor operably coupled to the processing device,
A smartwatch for detecting the presence of an abnormal health index of a user, comprising a memory operably coupled to the processing device, wherein the memory has a stored instruction and the instruction is: When run by a processing device,
The operation of receiving a raw PPG waveform signal from the PPG sensor and
The operation of receiving motion sensor data from the motion sensor and
The action of inputting the raw PPG waveform signal into a machine learning algorithm trained to detect whether the user's health index data is abnormal.
The operation of detecting that the health index data of the user is abnormal by the machine learning algorithm, and
To cause the processing device to perform an operation to generate a notification of the detected abnormal health index data.
Smart watch. The processing device
The operation of generating a heart rate waveform signal based on the raw PPG waveform, and
The smartwatch according to claim 1, further performing an operation of inputting the heart rate waveform signal together with the raw PPG waveform signal into the machine learning algorithm. The processing device
The operation of extracting the heart rate variability (“HRV”) signal from the raw PPG waveform and
The smart watch according to claim 1, further performing an operation of inputting the HRV waveform signal together with the raw PPG waveform into the machine learning algorithm. The processing device
The operation of extracting one or more features from the raw PPG waveform signal and
The smartwatch of claim 1, further performing the operation of inputting the one or more features together with the raw PPG waveform into the machine learning algorithm. It further comprises an ECG sensor coupled to the processing device, further comprising the ECG sensor to record ECG in response to the processing device detecting the presence of the abnormal health indicator data. , The smartwatch according to claim 1. Further comprising an ECG sensor coupled to the processing device, the processing device further comprises an action of notifying the user of the detected abnormal health index and an action of recording the ECG using the ECG sensor. The smartwatch according to claim 1, which is to be performed. The smart watch according to claim 1, wherein the abnormal health index data is an index of arrhythmia. A method for detecting a user's health index,
Depending on the processing device, the step of receiving a heart rate variability (HRV) signal from the PPG sensor,
A step of inputting the HRV signal by the processing device into a machine learning algorithm trained to detect whether the user's health index data is abnormal.
The step of detecting that the health index data of the user is abnormal by the machine learning algorithm, and
A method comprising the step of generating a notification of the detected abnormal health indicator data. The step of generating a heart rate signal based on the raw PPG waveform signal from the PPG sensor,
8. The method of claim 8, further comprising the step of inputting the heart rate waveform signal into the machine learning algorithm along with the raw PPG waveform signal. The step of extracting one or more features from the raw PPG waveform signal from the PPG sensor,
8. The method of claim 8, further comprising the step of inputting the one or more features into the machine learning algorithm along with the raw PPG waveform. The method of claim 8, further comprising recording ECG using an ECG sensor in response to detection of the presence of the abnormal health indicator data. The step of receiving a motion signal from the motion sensor,
8. The method of claim 8, further comprising inputting the motion signal into the machine learning algorithm trained to detect anomalous health index data. The method of claim 8, further comprising the step of notifying the user of the predicted abnormal health index data and recording the ECG using the ECG sensor. An abnormal health index, wherein the abnormal health index data is an index of arrhythmia. A non-temporary computer-readable storage medium containing an instruction that, when the instruction is executed by a processing device,
The operation of receiving a raw photopretismography (“PPG”) waveform signal from the PPG sensor, and
The operation of receiving motion sensor data from the motion sensor and
The action of inputting the raw PPG waveform signal into a machine learning algorithm trained to detect whether the user's health index data is abnormal, and
The operation of detecting that the health index data of the user is abnormal by the machine learning algorithm, and
To cause the processing device to perform an operation to generate a notification of the detected abnormal health index data.
Non-temporary computer-readable storage medium. The above command is
The operation of generating a heart rate waveform signal based on the raw PPG waveform signal, and
The non-temporary computer-readable storage medium of claim 15, further causing the processing device to perform an operation of inputting the heart rate waveform signal together with the raw PPG waveform signal into the machine learning algorithm. The above command is
The operation of extracting the heart rate variability (“HRV”) signal from the raw PPG waveform signal and
The non-temporary computer-readable storage medium of claim 15, further causing the processing device to perform an operation of inputting the HRV waveform signal together with the raw PPG waveform signal into the machine learning algorithm. The non-temporary computer-readable storage medium of claim 15, wherein the instruction causes the processing device to further record the ECG in response to detection of the presence of an arrhythmia. The non-temporary computer-readable memory of claim 15, wherein the instruction causes the processing device to further perform an action of notifying the user of the predicted arrhythmia and an action of recording the ECG using the ECG sensor. Medium. 15. The non-temporary computer-readable storage according to claim 15, wherein in order to input the motion sensor data into the machine learning algorithm, the instruction causes the processing device to further generate an activity level signal based on the motion sensor data. Medium.

Images