KR20210097512A - Method for predicting user heart state using active state of ppg and rest state of ppg - Google Patents

Abstract

A deep learning-based tumor treatment response prediction method is provided. According to one aspect of the present invention, a method for predicting a user's heart condition using photoplethysmogram in resting and active states includes the steps of: acquiring first PPG data in a resting state and second PPG data in an active state of a user by using a photoplethysmogram (PPG) sensor; inputting the first PPG data and the second PPG data to a first deep neural network and a second deep neural network, respectively, and calculating PPG analysis data through a first deep neural network and a second deep neural network; and predicting a user's heart condition based on the PPG analysis data and user's echocardiogram result data. Accordingly, it is possible to increase the reliability of a cardiac condition prediction result.

Description

A method for predicting a user's heart state using photoplethysmography waves in resting and active states

The present invention relates to a method for predicting a user's heart condition based on deep learning, and more particularly, to a method for predicting a user's heart condition based on the result of deep learning using photoplethysmography waves in resting and active states. .

Sudden death or sports death often occurs during exercise in healthy people. Most of these sudden deaths and sports events are considered accidents caused by latent diseases (cardiac diseases such as ischemic heart disease and angina pectoris), and in these cases, death usually occurs within 24-48 hours. It is important to prevent such accidents in advance by checking the presence or absence of potential diseases by measuring the exercise load test and determining the range of exercise.

In order to prevent unfortunate accidents caused by exercise, it is necessary to exercise appropriate to one's own physical strength. In order to set the exercise limit value of an exerciser and measure the appropriate amount of exercise, exercise load measurement is an essential condition. In particular, during continuous exercise after the age of 40, safety measures must be established through setting the exercise amount suitable for the athlete's physical strength and measuring the exercise load. And it is possible to prevent unfortunate accidents such as sports companies.

Methods for measuring the exercise load of an exerciser can be divided into two types. The first is isotonic exercise load measurement by dynamic exercise such as walking, running, cycling, etc., and the second is isometric exercise load, which only increases tension without changing the length of the muscle. In general, in isotonic exercise load, heart rate is correlated with oxygen uptake, and blood pressure x heart rate, which is an index of myocardial oxygen demand, is involved. Except for special cases, the isotonic exercise load test is often used for exercise load tests.

Commonly used factors to measure isotonic exercise load include heart rate, blood pressure, oxygen intake, ventilation, anaerobic threshold, respiratory exchange rate, resting metabolic rate, subjective exercise intensity, electrocardiogram, and target heart rate. The conventional exercise load measurement method mainly measures the change in heart rate during exercise using an electrocardiogram (ECG) in the case of heart rate, and the oxygen intake is measured by wearing a mask that measures respiration during exercise on the face.

However, this exercise load test requires expensive equipment to measure oxygen uptake and respiratory exchange rate, is inconvenient for the subject, such as the exerciser has to wear a face mask to exercise, and has disadvantages in that it is difficult to measure easily. In addition, as shown in Figure 1, the baseline electrocardiogram, the electrocardiogram of stage 1 (low or moderate intensity exercise), the electrocardiogram of stage 2 (high intensity exercise), and the electrocardiogram of the recovery period are different, which can be measured differently for each individual. . Moreover, the exercise load was a factor affecting the reflection of the exerciser's ground state by targeting the exerciser's arousal state and the exercise state, which caused a measurement error. In addition, since the exercise load test checks only the state of an athlete at a specific time, it is difficult to calculate an accurate state according to the overall lifestyle of each individual athlete.

(Patent Document 0001) KR 2009-0027390

SUMMARY OF THE INVENTION An object of the present invention is to provide a method for accurately predicting a user's heart condition by utilizing the photoplethysmography waves of the user's resting state and active state.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A method for predicting a user's heart state using a photoplethysmography wave in a resting state and an active state according to an aspect of the present invention for solving the above-described problems is a first method of predicting a user's resting state using a PPG (photoplethysmogram) sensor. acquiring PPG data and second PPG data of an active state; inputting the first PPG data and the second PPG data to a first deep neural network and a second deep neural network, respectively, and calculating PPG analysis data through the first deep neural network and the second deep neural network ; and predicting a user's heart condition based on the PPG analysis data and the user's echocardiogram data.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention as described above, it has various effects as follows.

According to the present invention, by utilizing the PPG data of the resting state and the active state, the user's lifestyle is reflected to increase the accuracy of predicting the heart state.

In addition, the present invention may increase the reliability of the cardiac condition prediction result by using the PPG analysis data and the echocardiography result data together.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is an exemplary view showing an electrocardiogram measured in a conventional exercise load test.
2 is a block diagram illustrating an apparatus for predicting a user's heart condition according to an embodiment of the present invention.
3 is a flowchart illustrating a method for predicting a user's heart condition according to an embodiment of the present invention.
4 is a flowchart illustrating a method of acquiring PPG data according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a method of acquiring PPG data according to an embodiment of the present invention.
6 is a block diagram illustrating a method for predicting a user's heart condition according to an embodiment of the present invention.
7 is a diagram illustrating a deep neural network according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term should be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating an apparatus for predicting a user's heart condition according to an embodiment of the present invention.

Referring to FIG. 2 , the apparatus 100 for predicting a user's heart condition according to an embodiment of the present invention may predict a user's heart condition by using the obtained PPG data. For example, the apparatus 100 for predicting a user's heart condition may output PPG analysis data using the PPG data of the resting state and the active state and the deep neural network, and the user's heart using the PPG analysis data and the echocardiogram result data. state can be predicted.

In an embodiment, the apparatus 100 for predicting a user's heart condition may be a wearable electronic device, and a dedicated program for setting a method for predicting a heart condition may be installed. For example, the apparatus 100 for predicting a user's heart condition obtains PPG data from the PPG sensor 110 and the PPG sensor 110 capable of detecting the PPG of the user, and performs echocardiography result data and other data from the external device 200 . Deep learning that calculates PPG analysis data through a data acquisition unit 120 that acquires data, etc., a data processing unit 130 that pre-processes (eg, normalizes) the acquired data, and a first deep neural network and a second deep neural network Unit 140, a heart condition determination unit 150 that predicts a user's heart condition based on PPG analysis data and echocardiography result data, and a database for storing PPG data, PPG analysis data, echocardiography result data, learning results, etc. 160 and a display 170 capable of displaying a message to the user.

For example, the PPG sensor 110 may be implemented while being separated from the main body of the smart watch and mounted on the watch strap. In this case, when the user wears the smart watch on the wrist, the PPG sensor mounted on the watch strap is in close contact with the user's wrist, thereby acquiring the user's photoplethysmogram by detecting the pulse on the wrist.

Also, the apparatus 100 for predicting a user's heart condition may audibly warn the user that the heart health condition is abnormal through a speaker.

In the apparatus for predicting a user's heart condition according to various embodiments of the present disclosure, the electronic device includes a smart phone, a tablet personal computer, a mobile phone, a video phone, and an e-book reader. ), desktop PC (desktop personal computer), laptop PC (laptop personal computer), netbook computer, PDA (personal digital assistant), PMP (portable multimedia player), MP3 player, mobile medical device, camera (camera) , or wearable devices (e.g., head-mounted-devices (HMDs) such as electronic glasses, electronic garments, electronic bracelets, electronic necklaces, electronic accessories, electronic tattoos, or smart watches. ) may include at least one of.

Also, the external device 200 may be a server or a medical device in which various types of medical data are stored.

3 is a flowchart illustrating a method for predicting a user's heart condition according to an embodiment of the present invention. 4 is a flowchart illustrating a method of acquiring PPG data according to an embodiment of the present invention. 5 is an exemplary diagram for explaining a method of acquiring PPG data according to an embodiment of the present invention. 6 is a block diagram illustrating a method for predicting a user's heart condition according to an embodiment of the present invention. 7 is a diagram illustrating a deep neural network according to an embodiment of the present invention.

3 to 7 , in an embodiment, in operation 31 , the PPG sensor 110 may acquire first PPG data in a resting state and second PPG data in an active state of the user, and the obtained second PPG data The first PPG data and the second PPG data may be transmitted to the data obtaining unit 120 . For example, the photoplethysmogram (PPG) uses the periodic change in blood flow in blood vessels as the heart beats. measurement to estimate the heart rate.

For example, the PPG sensor 110 may collect the measured bio-signals as learning data by controlling to measure the bio-signals at predetermined time intervals (eg, 15 minutes).

In an embodiment, the data acquisition unit 120 may classify and store the PPG data acquired according to the operation of FIG. 4 as first PPG data in a resting state or second PPG data in an active state. As shown in FIG. 5 , in the PPG data, the resting state data on the left and the active state data on the right are different from each other, so in order to accurately predict the heart state reflecting the user's lifestyle, the PPG data is divided into a resting state and an active state. need to learn

Specifically, in an embodiment, in operation 41 , the data acquisition unit 120 may acquire PPG data using the PPG sensor 110 .

In an embodiment, in operation 42 , the data acquisition unit 120 may check the length of the noise section of the PPG data, a preset period, or the heart rate of the user.

In an embodiment, in operation 43, the data acquisition unit 120 compares the PPG data with the first PPG data in the rest state and the first PPG data in the active state based on the length of the noise section of the PPG data or a preset period. 2 PPG data can be divided and saved. That is, when the length of the noise section of the PPG data is longer than the specified length, it can be determined that the user is in an active state, and when the length of the noise section of the PPG data is shorter than the specified length, it can be determined that the user is in a resting state. Alternatively, according to a preset period, the current user may be determined to be in a resting state, such as sleeping, or determined as an active state.

Alternatively, the data acquisition unit 120 may compare the user's heart rate with a preset heart rate range, and convert the PPG data into the first PPG data in the resting state and the second PPG data in the active state based on the comparison result. can be stored separately.

In an embodiment, in operation 32 , the data processing unit 130 may normalize the first PPG data and the second PPG data. For example, since it may be impossible to input the first PPG data and the second PPG data to the first deep neural network and the second deep neural network as the original data, a preprocessing operation is performed on the first PPG data and the second PPG data, Standardized learning data may be generated. That is, the function evaluation data and other data between the first PPG data and the second PPG data may be transformed into a standardized form (ie, standardized learning data) that may be input to the deep neural network through a preprocessing operation. For example, the first PPG data and the second PPG data may be converted into numerical values according to a predetermined rule, and the converted numerical values may be converted into vectors to be used as input values of the deep neural network. That is, the normalization operation refers to an operation of changing to suit the deep neural network within a range that does not affect the original meaning of the first PPG data and the second PPG data.

In an embodiment, in operation 33, the deep learning unit 140 may input the first PPG data and the second PPG data to the first deep neural network and the second deep neural network, respectively. Since the first PPG data and the second PPG data are different data representing a resting state and an active state, respectively, it may be advantageous in terms of accuracy to perform learning with different deep neural networks. For example, the first deep neural network and the second deep neural network may be a one-dimensional convolutioanal neural network (CNN) or a recurrent neural network (RNN). For example, the input values of the first deep neural network and the second deep neural network are the first PPG data and the second PPG data, respectively, and the output values may be respective analysis data, and the correct answer value is actually analyzed by a doctor or the like. It may be one data.

For example, in a recurrent neural network (RNN), one or more feature values (vectors) are input, and an output value of a hidden layer or an output layer for a previously input feature value is also input to a newly input feature value. A one-dimensional CNN (Convolutional Neural Network) receives a one-dimensional vector or matrix as an input, and a convolution step or pooling step can be performed before the general ANN step. In the convolution step, a multidimensional matrix with weights and a convolution operation are performed. In the pooling step, there are max pooling or mean pooling methods.

For example, in the case of a one-dimensional CNN, one-dimensional features can be more easily extracted from image data by using a one-dimensional filter with different weights. For example, if there is image data with different features for each frequency band, a one-dimensional filter with different weights for each frequency band for each time section is applied to the image data to extract features for frequency more precisely than 2D CNN. can

For example, when the first deep neural network is a one-dimensional CNN, as shown in FIG. 7 , the first deep neural network includes 11 convolutional layers, one fully-connected layer, It may include four max-pooling layers and a dropout layer. Here, the dropout layer may be disposed except for the first convolutional layer and the last convolutional layer. The convolutional layer may receive electrocardiogram data (1 x 3000 x L) and may have 64 or 32 filters with a size of 1 x 16. Convolutional layers are a layer that performs convolution on the input ECG signal, the max pooling layer can reduce the size of information by selecting the maximum value in a given region, and the drop-out layer is a layer of neurons in one layer to prevent overfitting. Only a certain percentage of the networks connected to neurons in the next layer can be selected. A fully connected layer can do transformations that connect both neurons in one layer and neurons in the next layer.

Meanwhile, the deep learning unit 140 may additionally input the user's demographic data and clinical data to the first deep neural network and the second deep neural network. For example, the heart condition prediction apparatus 100 may additionally acquire demographic data or clinical data of the user, and may predict the user's heart condition by additionally considering the demographic data and clinical data. For example, the demographic data may include at least one of patient gender, patient age, and household income. The clinical data may include at least one of family history, past medical history (eg, hypertension, diabetes, obesity, metabolic syndrome), drinking history, smoking history, physical activity, diet, lifestyle, height, weight, blood level, and cholesterol.

Meanwhile, the normalization process may be applied to demographic data and clinical data, like the PPG data described above.

On the other hand, unlike shown in the drawings, the user's demographic data and clinical data are not used as input values of the first deep neural network and the second deep neural network, but are directly input to the heart state determiner 150 to determine the heart state. It can be used as data for prediction.

In an embodiment, in operation 34, the deep learning unit 140 may calculate PPG analysis data through the first deep neural network and the second deep neural network. For example, the deep learning unit 140 combines a first result value output from the first deep neural network to which the first PPG data is input and a second result value output from the second deep neural network to which the second PPG data is input By concatenating, PPG analysis data can be calculated. Here, as the combination method, simple average, weighted average, majority voting, etc. may be applied.

For example, the PPG analysis data may include a user's heart rate, body temperature, stress level, and a degree of abnormality or distortion compared to a normal PPG waveform.

In an embodiment, in operation 35, the heart condition determiner 150 may predict the heart condition of the user based on the PPG analysis data and the user's echocardiogram result data. For example, the heart state determiner 150 may receive the user's echocardiogram result data from the external device 200 .

For example, echocardiography is a method of diagnosing a heart disease, in which an ultrasound signal is transmitted toward the heart using an ultrasound machine, and an image of a tomography of the heart is obtained from information of the ultrasound signal reflected from the tissues of the body. this is used At this time, analyzing the ultrasound image obtained from the ultrasound machine and displaying the tissue deformation characteristics such as contraction and contraction rate of each segment of the heart wall provides a direct and quantitative measure of the muscle's ability to contract and relax. Here, the external device 200 may be an ultrasound imaging device, irradiates an ultrasound signal from the body surface of an object toward a target region in the body, and uses information of the reflected ultrasound signal (ultrasound echo signal) to form a tomography or blood flow in the soft tissue. It may be a device for obtaining an image about the non-invasive.

For example, the echocardiography result data uses information of an ultrasound signal reflected from a tissue in the body received as an ultrasound signal is transmitted toward an object in the body, and an image of a tomography or blood flow of a soft tissue, and a degree of muscle contraction and It can contain the same characteristic value. In addition, the echocardiogram result data may include movement information of the myocardium. As the movement information of the myocardium, there may be exemplified myocardial strain, that is, myocardial strain, or distribution of myocardial strain, and myocardial stress.

As described above, the heart condition determination unit 150 determines whether or not a heart disease is present by comprehensively considering various information (eg, heart rate, blood flow image, muscle contractility, etc.) included in the PPG analysis data and the echocardiogram result data.

For example, the heart condition determination unit 150 may preset heart rate reference information including a lower limit threshold value and an upper limit threshold value by adding or subtracting a predetermined number based on the heart rate when the user is in a normal state, and if the user's heart rate is When the lower limit threshold is less than the upper threshold or exceeds the upper threshold, the user may determine that the user is in an emergency situation in which the heart health state is abnormal.

As another example, the heart rate reference information may include a unique change pattern of the user's heart rate and an allowable deviation range for the unique change pattern. In this case, when the change pattern of the user's heart rate is out of the allowed deviation range, the user may determine that the user is in an emergency situation in which the heart health state is abnormal.

Meanwhile, the heart condition prediction result output by the heart condition determination unit 150 is a probability value and may range from 0 to 1. If the result of the heart condition prediction is close to 1, the user may diagnose as having a heart disease, and if it is close to 0, it may be diagnosed as being in a normal state.

In an embodiment, in operation 36, the apparatus 100 for predicting a user's heart condition displays at least one of a user's heart function log and a heart disease occurrence probability through the display 170 based on the user's heart condition prediction result. can be displayed

For example, the display 170 may display a message “there was a change in heart function by how many percent compared to last month” corresponding to the heart function log, or “the likelihood of heart failure compared to the previous month” corresponding to the probability of occurrence of heart disease is displayed. has changed by how many percent".

As described above, the present invention can accurately predict the heart condition by reflecting the user's overall life.

According to an aspect of the present invention, a method for predicting a user's heart state using a photoplethysmography wave in a resting state and an active state is a method of predicting a user's heart state by using a PPG (photoplethysmography) sensor to determine the first PPG data in the user's resting state and the active state. 2 acquiring PPG data; inputting the first PPG data and the second PPG data to a first deep neural network and a second deep neural network, respectively, and calculating PPG analysis data through the first deep neural network and the second deep neural network ; and predicting a user's heart condition based on the PPG analysis data and the user's echocardiogram data.

According to various embodiments, at least one of a heart function log of the user and a probability of occurrence of a heart disease may be displayed on the display based on the result of the prediction of the user's heart condition.

According to various embodiments, the first deep neural network and the second deep neural network may be a one-dimensional convolutioanal neural network (CNN) or a recurrent neural network (RNN).

According to various embodiments, the method may further include normalizing the first PPG data and the second PPG data.

According to various embodiments, further acquiring demographic data or clinical data of the user; and predicting the heart condition of the user by further considering the demographic data and the clinical data.

According to various embodiments, the method may further include normalizing the demographic data and the clinical data.

According to various embodiments, obtaining PPG data using the PPG sensor; and dividing the PPG data into the first PPG data in the resting state and the second PPG data in the active state and storing the PPG data based on the length of the noise section of the PPG data or a preset period. can

According to various embodiments, obtaining PPG data using the PPG sensor; acquiring the heart rate of the user by analyzing the PPG data; comparing the heart rate of the user with a preset heart rate range; and dividing and storing the PPG data into the first PPG data in the resting state and the second PPG data in the active state based on the comparison result.

According to various embodiments, the method may further include receiving the echocardiogram result data from an external device.

According to various embodiments, the user's heart condition prediction program using the photoplethysmogram waves of the resting state and the active state is combined with a computer that is hardware, and is stored in a medium to execute the method of any one of claims 1 to 9. can be

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: user heart condition prediction device
110: PPG sensor
120: data acquisition unit
130: data processing unit
140: deep learning unit
150: heart condition determination unit
160 : database
170: display

Claims (10)

A method for predicting a user's heart condition using photoplethysmography waves in resting and active states,
obtaining first PPG data in a resting state and second PPG data in an active state of the user by using a photoplethysmography (PPG) sensor;
inputting the first PPG data and the second PPG data to a first deep neural network and a second deep neural network, respectively, and calculating PPG analysis data through the first deep neural network and the second deep neural network ; and
Predicting the user's heart condition based on the PPG analysis data and the user's echocardiogram result data; The light volume of the resting state and the active state according to claim 1, wherein at least one of a heart function log of the user and a probability of occurrence of a heart disease is displayed on a display based on a result of the prediction of the user's heart condition. A method for predicting the user's heart condition using pulse waves. According to claim 1, wherein the first deep neural network and the second deep neural network is a one-dimensional CNN (convolutioanal neural network) or RNN (Recurrent Neural Network) characterized in that the photoplethysmogram wave of the resting state and the active state A method for predicting the user's heart condition. The method of claim 1, further comprising normalizing the first PPG data and the second PPG data. The method of claim 1 , further comprising: acquiring demographic data or clinical data of the user; and
The method of predicting a user's heart condition using the photoplethysmography waves of the resting state and the active state, further comprising the step of predicting the heart condition of the user by additionally considering the demographic data and the clinical data. [6] The method of claim 5, further comprising normalizing the demographic data and the clinical data. According to claim 1,
acquiring PPG data using the PPG sensor; and
Separating the PPG data into the first PPG data in the resting state and the second PPG data in the active state and storing the PPG data based on the length of the noise section of the PPG data or a preset period. A method for predicting the user's heart condition using the photoplethysmography waves of the resting state and the active state. According to claim 1,
acquiring PPG data using the PPG sensor;
acquiring the heart rate of the user by analyzing the PPG data;
comparing the heart rate of the user with a preset heart rate range; and
Based on the comparison result, dividing and storing the PPG data into the first PPG data in the resting state and the second PPG data in the active state; A method for predicting the user's heart condition using photoplethysmography. The method of claim 1, further comprising receiving the echocardiogram result data from an external device. A user heart condition prediction program using photoplethysmography waves in resting and active states, stored in a medium to execute the method of any one of claims 1 to 9 in combination with a computer that is hardware.

Images