KR102306115B1 - A Novel Healthcare Monitoring Method and Apparatus Using Wearable Sensors and Social Networking Data - Google Patents

Abstract

A healthcare monitoring method and device using wearable sensors and social network data are presented. The healthcare monitoring device using the wearable sensor and social network data proposed in the present invention provides physiological data from wearable devices, drug and symptom data from smart terminals, and application programming interfaces (APIs) for patients and doctors. A data collection unit that collects discussion data on social networks, a data storage unit that connects to a personal cloud server to store collected physiological data, drug and symptom data, and discussion data, and feature polarity identification, word embedding and ontology for the stored data It includes a big data analysis engine that performs calculation and classification of data, including basic feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and drug side effects prediction.

Description

A Novel Healthcare Monitoring Method and Apparatus Using Wearable Sensors and Social Networking Data

The present invention relates to a healthcare monitoring method and apparatus using a wearable sensor and social network data.

Current technology in the medical industry plays a key role in providing efficient methods for collecting physiological information about individual patients. In addition, efficient medical monitoring is possible by collecting patient data using smartphones and wearable sensors. People also use social networking to share their feelings and opinions about certain drugs, which is useful for observing emotional disorders and predicting drug side effects. However, continuous patient monitoring generates large amounts of medical data, and medical data generated by users of social networking sites can be massive, unstructured, and unexpected. The current trend therefore calls for advanced approaches capable of processing vast amounts of medical data extracted from a variety of sources, including smartphones, wearable sensors, medical records, and social networks (i.e. big data). Current medical monitoring systems are not efficient at extracting sensitive information from device and social networking data, and they struggle to effectively analyze them. In addition, existing machine learning techniques cannot handle and process extracted medical data for abnormal prediction. Therefore, we propose a new healthcare monitoring architecture based on cloud environment and big data analysis engine to precisely store and analyze medical data and improve classification accuracy.

An object of the present invention is to provide a new healthcare monitoring method and apparatus based on a cloud environment and a big data analysis engine to precisely store and analyze medical data and improve classification accuracy.

In one aspect, the health care monitoring device using the wearable sensor and social network data proposed in the present invention is physiological data from a wearable device, drug and symptom data from a smart terminal, and application programming interfaces (APIs) through A data collection unit for collecting discussion data on social networks of patients and doctors, a data storage unit for storing collected physiological data in connection with a personal cloud server, drug and symptom data, and discussion data, and feature polarity for the stored data Includes a big data analytics engine that performs computation and classification of data including identification, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and drug side effects prediction do.

The big data analytics engine stores data including sensor data, medical records and social network content, and includes pre-analysis of sensor data, pre-processing and filtering of sensor data, pre-processing of medical records, pre-processing of social network content, and inconsistencies After performing data filtering to remove noise, the data is converted into a structured form.

The big data analysis engine retrieves all ontology information by performing ontology-based feature extraction to provide additional information about the neural network model that understands the meaning of words, integrates with word embeddings for classification, and extracts information from the ontology. For this purpose, it is mathematically defined using a statistical method including Term Frequency (TF), Term Frequency, and TF-IDF (Inverse Document Frequency).

The big data analysis engine uses the information gain (IG) method calculated through WEKA (Waikato Environment for Knowledge Analysis) to extract the features of the wearable sensor data, find the value of the attribute, and calculate the entropy to measure the uncertainty. and define the difference between the entropy before and after entropy.

In another aspect, the healthcare monitoring method using the wearable sensor and social network data proposed in the present invention includes physiological data from a wearable device through a data collection unit, drug and symptom data from a smart terminal, and an application programming interface (Application Programming Interface). Interfaces; collecting discussion data on social networks of patients and doctors through API), the data storage unit is connected to a personal cloud server to store collected physiological data, drug and symptom data, and discussion data, and Data is analyzed through big data analysis engine to identify feature polarity, word embedding and ontology-based feature extraction, extract from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and calculation and prediction of drug side effects. performing classification.

According to the embodiments of the present invention, medical data can be efficiently pre-processed through data mining technology based on data mining technology, ontology, and bi-directional long-term memory (Bi-LSTM) through the proposed big data analysis engine and useful functions are provided. It can be extracted and reduced the number of dimensions of the data. In addition, the proposed ontology provides semantic knowledge about entities and aspects and relationships in the diabetes and blood pressure domains, and Bi-LSTM can accurately classify medical data to predict drug side effects and abnormal states of patients. In addition, the proposed device is utilized for patient classification using medical data related to diabetes, BP, mental health and drug review, to accurately process heterogeneous data through the proposed model and improve the accuracy of health status classification and drug side effects prediction can do it

1 is a diagram showing the configuration of a healthcare monitoring device using a wearable sensor and social network data according to an embodiment of the present invention.
2 is a diagram for explaining a big data cloud server and HDFS according to an embodiment of the present invention.
3 is a diagram for explaining a big data analysis engine according to an embodiment of the present invention.
4 is a diagram for explaining embedding and ontology-based feature extraction according to an embodiment of the present invention.
5 is a diagram for explaining LSTM-based classification and prediction according to an embodiment of the present invention.
6 is a flowchart illustrating a healthcare monitoring method using a wearable sensor and social network data according to an embodiment of the present invention.
7 is a graph showing the accuracy and MAE of LSTM-based classification according to the use of ontology according to an embodiment of the present invention.
8 is a graph showing results obtained by using a classification model together with PCA and IG according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing the configuration of a healthcare monitoring device using a wearable sensor and social network data according to an embodiment of the present invention.

The proposed healthcare monitoring device using the wearable sensor and social network data includes a data collection unit 120 , a data storage unit 130 , and a big data analysis engine 140 .

The data collection unit 120 collects physiological data from a wearable device, drug and symptom data from a smart terminal, and discussion data of patients and doctors on social networks through application programming interfaces (APIs).

For example, the data collection unit 120 may collect data from the data source 110 including the sensor 111 , the hospital medical record 1120 , the hospital SNS 113 , and the patient SNS 114 . In other words, it collects physiological information from wearable devices, drug and symptom information from smartphones, and social network discussions of patients and doctors through application programming interfaces (APIs).

Wearable devices use smart sensors such as blood pressure monitors, glucometer sensors, smart watches, pulse oximeters, accelerometers, temperature sensors, and weight scales and wearable devices to measure blood pressure, blood sugar level, heart rate, stress ratio, oxygen saturation, temperature and weight. The physiological signs of the patient can be monitored in real time.

Smartphones are used to collect dietary, exercise and other activity information from patients with diabetes and BP, and to collect personal information (eg age, gender, height, other information).

Medical records include a patient's medical record that describes the patient's medical history (eg, treatment, laboratory tests, drug intake).

Data from social networking platforms and web pages collect social media data, such as drug reviews or patient emotional posts, to predict levels of stress and depression, identify side effects of diabetes medications in the context of diet and lifestyle, and improve patient care and intervention. Improve your knowledge.

The data storage unit 130 is connected to the personal cloud server in this way and stores the collected physiological data, drug and symptom data, and discussion data.

As described above, large amounts of medical data collected from various sources are difficult to store and process. Therefore, the proposed device connects to a private cloud server called Amazon S3, which is highly scalable and highly secure, to store patient information. A data storage process will be described in more detail with reference to FIG. 2 .

2 is a diagram for explaining a big data cloud server and HDFS according to an embodiment of the present invention.

As shown in FIG. 2 , the wearable sensor and social networking data are transmitted from the cloud server 210 to the Hadoop Distributed File System (HDFS) 232 . HDFS (232) is a distributed file storage system that provides high bandwidth to transfer data to the HBase (231) cluster. HBase 231 runs on top of HDFS 232 . Flume 220 transfers data from the cloud server to the Hadoop ecosystem.

According to an embodiment of the present invention, Apache Pig 242 is used to transmit the data extracted by the wearable device to the MapReduce 242. MapReduce(242) has two main functions. That is, Map and Reduce. The Map task collects values from big data of a pair of key values. The Reduce function saves the set value for a specific key. Thereafter, the stored data is transmitted to the analysis engine (Analytics Engine) 250 .

Big data analysis engine 140 identifies feature polarity for stored data, ontology-based semantic knowledge (word embedding and ontology-based feature extraction from 1430), extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP Perform calculation and classification of data including classification and prediction of drug side effects.Stored data consists of sensor data, medical records and social networking data.But inconsistencies, missing values, noise, different formats, large size, high dimension Because of the nature, it is extremely difficult to process real big data.

The big data analysis engine 140 is the most important component of the proposed framework. The big data analysis engine 140 is divided into two lower layers: a data calculation unit 141 and a data classification unit 142 . The data calculation unit 141 includes sub-modules such as data preprocessing 141a, data dictionary analysis 141b, feature extraction 141c, word dimension reduction 141d, and word embedding 141e. Here, the ontology-based semantic knowledge 143 is used to process and analyze data necessary for extraction of necessary information, together with a soft computing approach. The data classification unit 142 uses the Bi-LSTM 142a with ontology to classify diabetes, BP, mental health, and drug side effects. The data classification unit 142 intelligently analyzes multidimensional big data related to diabetes and BP to obtain insight into decision making from the data, and provides a customized diabetes and BP medical system to the patient. A data analysis process will be described in more detail with reference to FIG. 3 .

3 is a diagram for explaining a big data analysis engine according to an embodiment of the present invention.

3 , the big data analysis engine performs pre-analysis of sensor data, pre-processing and filtering of sensor data, pre-processing of medical records, and pre-processing of social network contents.

resource ID parameter Explanation



sensor S1 blood sugar Blood sugar (mg/dL) S2 temperature patient's current body temperature S3 Blood pressure Systolic blood pressure (mmHg) diastolic blood pressure (mmHg) S4 oxygen saturation Patient SpO ₂ Consumption (mmHg) S5 heart rate Patient's heart rate (bpm) S6 ECG Patient's electrocardiogram using ECG sensor S7 EEG Patient EEG examination using EEG sensor S8 stress Calculation of patient stress using ECG + EEG pattern


Smartphone SP1 age date of birth patient age SP2 key necropsy to find the patient's height SP3 BMI body mass index (kg/m ² ) SP4 gender Patient's gender (0/1) SP6 activity Lifestyle: sedentary, light-active, moderately active, active or very active







hospital medical records MR1 lipoprotein level Low-density lipoprotein levels (LDL cholesterol) High-density lipoprotein levels (HDL cholesterol) MR 2 hemoglobin Patient's glycohemoglobin (A1c) (%) MR 3 blood sugar patient's blood test MR 4 serum creatinine patient's blood test MR5 triglycerides patient's blood test MR6 cholesterol patient's blood test MR7 AST (SGOT) blood tests to check for liver damage MR8 ALT (SGPT) blood tests to check for liver damage MR9 taking drugs extracted from the prescription list MR10 smoking Yes / No (Self Test) MR11 Drinking Yes / No (Self Test) MR12 Indigestion Yes / No (Self Test) MR13 family history Yes / No (Self Test) MR14 patient's medical history Yes / No (Self Test)

In the preliminary analysis of sensor data, as shown in Table 1, different parameters are detected from diabetic and BP patients using sensors and smartphones. It also extracts other parameters from the patient's body. First, the data set is converted to a comma-separated values (CSV) file for easy parsing. A name is assigned to each property (parameter) and is displayed in the form of a column with a numeric value. Then, the ID of the sensor data is displayed as the actual sensor name (for example, S1 is the blood sugar ID). Instead of using explicit years and hours for age and activity, respectively, divide the accelerated age into days, months, and years, and divide activity hours into morning, afternoon, and evening. These generated features provide insight into patient health, such as patterns of normal and abnormal conditions depending on whether a serious condition occurs at the beginning or end of a month, or during a start month and end month. The final data set is then uploaded to the Hadoop cloud environment for further processing.

The sensor data is pre-processed and filtered prior to analysis. Filter the data to remove inconsistencies and noise. Clean up data by removing ASCII characters. To remove this noise from the data, we use a well-known filtering approach called Kalman filtering. Also, an unsupervised filter called ReplaceMissingValues is used to replace all missing numeric values in the data set with the means and modes of available data. Useless attributes are removed with a variance of up to 90% using an unsupervised filter called RemoveUseless filter. The numeric values are then normalized using a normalization filter, constrained to between 0 and 1 for all classifications. After these steps, we utilize EmEditor to split the data set into n data files and upload them to the Hadoop cloud environment for further processing.

In the pre-processing of the Medical Record (MR), information on drugs included in the prescribed drug data for diagnosis is used. Some of the characteristics of MR can be used to classify patients. Therefore, for data analysis, ID and reference values (0, 1, 2) are assigned to each attribute of MR. The reference values 0, 1, and 2 represent the patient's health status (normal, pre-diabetes, diabetes). MR data is also used to overcome the limitations of sensor-based generated data. For example, replace missing values with the current MR attribute values in the data set.

In the pre-processing of social network content, the following steps are applied to transform the data into a structured form. These steps also remove useful data, which helps to easily extract features and opinion words.

First, stop word removal is performed. Words such as prepositions (to, in, and of), all articles (a, an, and the), symbols (#, @, etc.), and URLs of corpus data do not interfere with the meaning of the document. However, the accuracy of text classification decreases. This content is deleted using a well-known method called Rainbow according to an embodiment of the present invention.

Then, in the tokenization stage, white spaces and delimiters are removed to separate complex texts in the corpus into small terms or tokens. Apply an n-gram tokenizer to remove white spaces and delimiters. The output is then stored for further analysis such as speech (PoS) tagging of the extracted words and headwords.

In the PoS tagging step, the corpus text is divided into sentences, and then Stanford Core Natural Language Processing (CoreNLP) is used for POS tagging.

In the stemming and lemmatization steps, the present invention applies a suffix dropping algorithm for stem extraction. Headword extraction expresses the headings of words used in the text. After extracting the headword, it is easy to obtain the lexical information of each word. For example, blood sugar is related to blood glucose. Thus, the stem and headword words are utilized for further processing.

In the feature transformation phase, the patient uses unusual words (eg, depressssssed) in the SNS that affect the classifier result. Therefore, a series of characters appearing more than once is converted to a general word (for example, depressssed becomes depressed).

Next, feature polarity identification is performed. The proposed device uses content posted on social networks to detect stress and depression in patients. They also undertake several drug reviews to understand their opinions on the effectiveness and side effects of diabetes medications. A sentiment analysis approach is used for the two previously mentioned tasks. Therefore, it is important to find feature polarity and document labeling for sentiment classification. After preprocessing social networks and web page content, SentiWordNet (SWN) is used to determine feature opinion word polarity. Then, the result of feature polarity is accumulated to find the polarity of the entire document. Then, word embedding and ontology-based feature extraction are performed. An embedding and ontology-based feature extraction process will be described in more detail with reference to FIG. 4 .

4 is a diagram for explaining embedding and ontology-based feature extraction according to an embodiment of the present invention.

As shown in Figure 4, a skip-gram model of Word2vec's skip program with 200-dimensional vectors in the data set was trained. This Word2vec model is used to train an ML classifier to detect associations between features. In general, ML classifiers miss the basic semantics of features depending on a particular domain. An ontology may indicate the semantics of a given domain in some cases. Biomedical Ontologies, available online, cover a variety of topics related to diabetes, depression, high blood pressure, medications, and food.

Ontologies and Abbreviations Justice number of classes number of features number of individuals Nutrition Ontology (ONS) Provides a description of complex nutrition research. 3442 66 104 BioMedBridges Diabetes Ontology (DIAB) Represents the relationship between diabetes phenotypes for text mining. 375 4 0 Diabetes Treatment Ontology (DMTO) Provides interoperability for the treatment of diabetes. 10700 315 63 Human Disease Ontology (DOID) It represents the concept of a rare disease. 12694 15 0 Drug Target Ontology (DTO) It provides information on the classification of drug target data. 10075 0 0 FHIR and SSN-based Type 1 Diabetes Ontology (FASTO) Insulin management information for people with diabetes. 9577 822 460

As shown in Table 2, Nutrition Research Ontology (ONS), BioMedBridges Diabetes Ontology (DIAB), Diabetes Mellitus Treatment Ontology (DMTO), Human Disease Ontology (DOID), Drug Target Ontology (DTO), Fast Healthcare Interoperability Utilizes the latest edition of Type 1 Diabetes Ontology (FASTO) based on Resource (FHIR) and Semantic Sensor Network (SSN). These ontology provide additional information for neural network models that understand the meaning of unusual words.

It retrieves all ontology information and attempts to integrate it with word embeddings for the classifier. Think of each ontology as BOW (Bag-of-Words). To extract information from an ontology, well-known statistical methods called term frequency (TF), term frequency and TF-IDF (Inverse Document Frequency) are used. Here, each concept or characteristic of ontology is considered as a term, and ontology is considered as a document. Therefore, TF is a term (peripheral word) found in ontology. TF is mathematically defined in the following equation.

(1)

(One)

If the TF(Term, Onto) value is greater than 0, repeat this step to extract a specific concept. The TF-IDF selects an example term for information extraction. IDF mainly reduces the meaning of words appearing in all ontology (eg, patient, disease, hospital, etc.). If a term occurs in more ontologies or more concepts in a single ontology, it means that it is a regular term and may not be a term necessary for information extraction. Therefore, the result of the log function of the input word will be reduced to zero. This shows that the value of TF-IDF is small compared to this term. The statistical description of IDF is shown in the following equation.

(2)

(2)

는 데이터베이스의 전체 온톨로지 수 또는 온톨로지 내의 총 개념 수를 표시한다(예를 들어,

및 |{Onto∈Ontos:Term ∈Onto}|는 용어가 나타나는 온톨로지 내의 온톨로지 또는 개념의 수이다). 다음 TF-IDF 수학식을 사용하여 공통 용어를 제거한다.here

indicates the total number of ontology in the database or the total number of concepts in the ontology (for example,

and |{Onto∈Ontos:Term ∈Onto}| is the number of ontology or concept in the ontology in which the term appears). The following TF-IDF equations are used to remove common terms.

(3)

(3)

The TF-IDF result shows how essential the ontology function is to the ontology in the ontology corpus or how important the ontology function is to the concept in the ontology.

Next, features extracted from wearable sensor data will be described. We use the information gain (IG) method for feature extraction, which affects the classifier by reducing features that are not related to noise. The information gain is calculated using Waikato Environment for Knowledge Analysis (WEKA). The results are obtained by applying the information gain filter "infoGainAtrributeVal" as an evaluator for the diabetes and BP data sets. Once the attribute values are identified, the IG measurement is associated with a reduction in entropy of the training data set. This approach finds the value of an attribute by calculating the IG according to its classification. The proposed IG method uses entropy to measure system uncertainty and finds the difference between the previous entropy and the subsequent entropy. It specifies the amount of additional information about A provided by B as in equation (4).

(4)

(4)

Here, A and B are separate variables. A is the feature, and the previous entropy can be measured using Equation (5).

(5)

(5)

는

의 이산 값에 대한 사전 확률을 나타낸다. A의 조건부 엔트로피는, 이후의 엔트로피 B가 주어진 후에, 수학식 6과 7에서와 같이 정의될 수 있다.here

Is

Represents the prior probabilities for the discrete values of . The conditional entropy of A can be defined as in Equations 6 and 7 after the entropy B is given.

(6)

(6)

(7)

(7)

The information gain can be calculated by putting Equations 5 and 7 into Equation 4 as in Equation 8.

(8)

(8)

Next, the main components are analyzed. The number of dimensions in the data set increases after pre-processing and creation of additional attributes. This causes problems such as reduction of classification accuracy, goodness of fit, and time complexity. Therefore, we use principal component analysis (PCA), a statistical approach to dimensionality reduction. PCA converts p-dimensional X data into q-dimensional Y data with the least loss.

Then, Bi-LSTM-based diabetes and BP classification and drug side effects are predicted. Bi-LSTM is used to classify patients' diabetes, BP, mental health, and side effects of diabetes medications.

5 is a diagram for explaining LSTM-based classification and prediction according to an embodiment of the present invention.

5 shows the structure of Bi-LSTM for classification of diabetes, BP, mental health, and drug side effects. Apply the grid search optimization algorithm to identify optimal values for the hyperparameters of the LSTM model. The optimal values chosen for hyperparameter dropout rate, epochs, batch size and learning rate are 0.3, 30, 32, and 0.001, respectively. As shown in Figure 5, LSTM is used to classify four different types of data sets. For diabetes classification, the Pima Indians Diabaetes dataset was collected from the UCI machine learning repository. The data set has eight input attributes. However, only six attributes are used to train the diabetes classification model. These attributes are age, BMI, BP, plasma glucose, diabetic lineage function, and 2-hour serum insulin. A BP classification model was trained using the PhysioNet multi-parameter intelligent monitoring dataset from the Intensive Care II (MIMIC-II) database. This data set consists of multiple samples of vital signs, including BP and heart rate (HR). Data sets for predicting drug side effects were obtained from the UCI repository. This data set consists of six properties. However, only two attributes are considered in the proposed work: drug and patient review name. In the present invention, we developed four Bi-LSTM-based classifier models: ^{LSTM a} , LSTM ^b , LSTM ^c , and LSTM ^d.

Diabetes classification uses age, family, gender, activity, BMI, blood pressure, and blood glucose functions. Enter each time step, t, and characteristic Xt in LSTM ^a units to predict diabetes grade {normal, pre-diabetic, diabetic}. The medical rules for classifying diabetic and blood pressure patients are presented in Table 3.

In blood pressure and ECG tests, wearable sensors are used to identify each patient's BP and heart rate. The extracted information is used for feature (shape) construction. After feature generation and preprocessing, ^{we developed a Bi-LSTM-based model named LSTM b} , which can maintain temporal patterns present in past sequenced data. For classification of BP, all parameters and features generated in each step t are converted into category vectors {0, 1, 2}.

In mental health monitoring, patient messages and posts are extracted from social networks and filtered using supervised and unsupervised approaches. After training a 200-dimensional Word2vec model, word sequences were fed to the LSTMc model. The ^{output of LSTM c} is then assigned to a softmax function, which predicts a sentiment label (positive, neutral, negative or strong negative) for each sentence in the published content. The rules for classifying a patient's mental health based on predicted polarity are shown in Table 4.

category Emotional polarity of patient posts and comments
(positive/neutral/negative/
strong voice) Emotional polarity to antidiabetic drugs
(positive/neutral/negative)
Patient Mental Health Classification happiness positivity - normal neutrality - depressed voice - stress strong voice -
Predicting drug side effects No side effects - positivity moderate side effects - neutrality serious side effects - voice

In the drug side effect prediction, current diabetes and BP drugs were used as input queries, and reviews were searched for on other websites. After filtering and preprocessing, the review is automatically labeled as no side effects/moderate side effects/serious side effects. For this labeling, identify the emotional polarity of each sentence. Then, as shown in Table 4, positive, neutral, and negative emotional polarities are considered to have no side effects, respectively, and no moderate side effects and no serious side effects. A 200-dimensional Word2vec model was trained to represent drug reviews as a series of words in the ^{LSTM d model.} We then ^{feed the output of LSTM d to} the softmax function that accurately predicts drug side effects.

Referring back to FIG. 1 , analysis data according to the data calculation and classification results are displayed through the data display unit 150 for healthcare monitoring 151 and recommendation 152 . The results from the diabetes classification are presented in Table 5.

Proposed model and other classifiers Degree (P) (%) Recall (R) (%) Functional Measurement (FM) (%) Accuracy (Ac) (%) RMSE MAE CNN 62 66 63 66 58 34 MLP 67 67 67 68 51 34 SVM 67 70 68 70 54 30 Fuzzy Classifier 73 72 65 72 52 26 logistic regression 70 68 69 69 55 32 random forest 67 70 66 70 46 42 KNN 57 65 59 65 52 50 LSTM ^a 74 75 75 75 50 26

Other classifiers (CNN, MLP, SVM, fuzzy logic, logistic regression, random forest, and KNN) are compared with the LSTMa model proposed using the Pima Indians data set. became As a result, the proposed LSTM ^a obtained the highest accuracy (75%) and lowest MAE (26%) compared to other classifiers. This high accuracy indicates that LSTM ^a stores more important information in memory cells for diabetes classification. Also, the lowest MAE shows better performance of the ^{LSTM a model.} The fuzzy classifier obtained the lowest RMSE (46) compared to other classifiers. Based on this experiment, we observe that other classifiers are time consuming and degrade performance even with a small number of features. However, LSTM outperformed other classifiers in predicting diabetes.

Table 6 shows the results obtained by the proposed model and other classifiers in terms of BP classification.

Proposed model and other classifiers Degree (P) (%) Recall (R) (%) Functional Measurement (FM) (%) Accuracy (Ac) (%) RMSE MAE CNN 71 70 69 70 54 30 MLP 80 80 80 80 37 22 SVM 83 75 73 74 50 25 Fuzzy Classifier 84 83 83 83 40 16 logistic regression 72 72 71 72 51 29 random forest 78 70 67 70 45 42 KNN 60 58 56 58 64 42 LSTM ^b 89 87 87 88 34 12

LSTM ^b is the most effective compared to CNN (70%), MLP (80%), SVM (73%), fuzzy classifier (83%), logistic regression (72%), random forest (70%), and KNN (58%). It was observed to achieve high accuracy (88%). However, the MAE and RMSE of the proposed model were 12 and 34, respectively, which were lower than other classifiers.

Table 7 shows the performance of the classifier in terms of mental health classification.

Proposed model and other classifiers Degree (P) (%) Recall (R) (%) Functional Measurement (FM) (%) Accuracy (Ac) (%) RMSE MAE CNN 66 66 66 66 56 34 MLP 72 71 71 72 47 29 SVM 68 68 68 68 56 32 Fuzzy Classifier 60 58 56 59 67 45 logistic regression 70 75 72 71 44 34 random forest 60 60 59 60 58 40 KNN 64 63 64 64 60 36 LSTM ^c 87 90 87 89 35 15

According to Table 7, LSTM ^c using Softmax shows the highest accuracy (89%) compared to other classifiers. Also, ^{the MAE and RMSE of LSTM c} are 15 and 35, respectively, which are lower than other classifiers.

Proposed model and other classifiers Degree (P) (%) Recall (R) (%) Functional Measurement (FM) (%) Accuracy (Ac) (%) RMSE MAE CNN 76 68 65 68 62 39 MLP 82 81 81 81 39 20 SVM 82 82 82 82 36 18 Fuzzy Classifier 79 78 78 78 41 28 logistic regression 84 82 83 83 36 17 random forest 76 66 63 66 58 33 KNN 77 75 75 76 49 24 LSTM ^d 88 90 89 90 32 13

To evaluate the classification results using the drug review of the proposed model, the LSTM ^d model was compared with other classifiers as shown in Table 8, and logistic regression analysis obtained high accuracy in 90% and 83%, respectively. Other classifiers achieved lower accuracy compared to ^{LSTM d and logistic regression.} The RMSE and MAE of LSTM ^d are 32 and 13, respectively, which are lower than other classifiers. CNN's accuracy is very low, and its RMSE is very high compared to other classifiers. This indicates that LSTM ^d can handle long sequences of word vectors as the size of the word vectors increases. In contrast, CNNs become overfitting as the dimensionality of word order and word embeddings increases.

6 is a flowchart illustrating a healthcare monitoring method using a wearable sensor and social network data according to an embodiment of the present invention.

The proposed healthcare monitoring method using wearable sensors and social network data includes physiological data from wearable devices through the data collection unit, drug and symptom data from smart terminals, and patients and doctors through application programming interfaces (API). Collecting discussion data on their social networks (610), the data storage unit is connected to a personal cloud server to store the collected physiological data, drug and symptom data, and discussion data (620), the stored data is big data analysis Calculation and classification of data including feature polarity identification, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and drug side effects prediction through the engine and ( 630 ) and displaying ( 640 ) analysis data according to the results of data calculation and classification.

In step 610, physiological data from wearable devices, drug and symptom data from smart terminals, and discussion data of patients and doctors on social networks through application programming interfaces (APIs) are collected through the data collection unit. .

In step 620, the data storage unit is connected to the personal cloud server to store the collected physiological data, drug and symptom data, and discussion data.

Stored data includes sensor data, medical records and social network content, for pre-analysis of sensor data, pre-processing and filtering of sensor data, pre-processing of medical records, pre-processing of social network content and for removing inconsistencies and noise After performing data filtering, the data is converted into a structured form.

To provide additional information about the neural network model that understands the meaning of words, perform ontology-based feature extraction to retrieve all ontology information, integrate it with word embeddings for classification, and extract information from the ontology, term frequency ( It is defined mathematically using statistical methods including Term Frequency (TF), term frequency, and TF-IDF (Inverse Document Frequency).

Using the information gain (IG) method calculated through Waikato Environment for Knowledge Analysis (WEKA), we extract features of wearable sensor data and find the value of the attributes, and use entropy to measure uncertainty and previously entropy and the entropy difference thereafter.

In step 630, the stored data is identified through the big data analysis engine, feature polarity identification, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and drug side effects prediction Calculation and classification of data including

In step 640, the analysis data according to the data calculation and classification results are displayed.

7 is a graph showing the accuracy and MAE of LSTM-based classification according to the use of ontology according to an embodiment of the present invention.

Fig. 7 shows the accuracy and MAE of ^{the proposed LSTM c} and LSTM ^d models without ontology/using ontology. As can be seen, the proposed ontology LSTM shows a significant improvement over the simple LSTM. In the classification of sentimental characters, ^{the accuracy of LSTM c} was increased to 84% and to 87% when the ontology was used. However, the MAE was reduced by only one. ^{In addition, the accuracy and MAE of LSTM d} were 90% and 14, respectively, in the classification of drug side effects. However, ^{when the ontology was used together with LSTM d} , the accuracy increased to 93%, and the MAE decreased by 1.

8 is a graph showing results obtained by using a classification model together with PCA and IG according to an embodiment of the present invention.

The results obtained using the classification model with PCA and IG are shown in FIG. 8 . Note that, based on the results obtained in FIG. 8 , ^{the accuracy of LSTM a} , LSTM ^b , ontology- ^{LSTM c} and ontology-LSTM ^d using PCA and IG increased by 4%, 1%, 3%, and 1%, respectively.

According to the embodiments of the present invention, medical data can be efficiently pre-processed through data mining technology based on data mining technology, ontology, and bi-directional long-term memory (Bi-LSTM) through the proposed big data analysis engine and useful functions are provided. It can be extracted and reduced the number of dimensions of the data. In addition, the proposed ontology provides semantic knowledge about entities and aspects and relationships in the diabetes and blood pressure domains, and Bi-LSTM can accurately classify medical data to predict drug side effects and abnormal states of patients. In addition, the proposed device is utilized for patient classification using medical data related to diabetes, BP, mental health and drug review, to accurately process heterogeneous data through the proposed model and improve the accuracy of health status classification and drug side effects prediction can do it

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (8)

a data collection unit that collects physiological data from wearable devices, drug and symptom data from smart terminals, and discussion data on social networks of patients and doctors through application programming interfaces (APIs);
a data storage unit for storing collected physiological data, drug and symptom data, and discussion data in connection with a personal cloud server; and
Calculation and classification of data including feature polarity identification for stored data, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and drug side effects prediction Big data analytics engine
including,
data storage unit,
It receives wearable sensor data, social network content, and web page content from a personal cloud server and transmits it to HDFS (Hadoop Distributed File System) through Flume. It is a distributed file storage system that provides bandwidth exceeding that of HDFS for transfer, uses Apache Pig to transfer wearable sensor data, social network content, and web page content to MapReduce, and paired key values through MapReduce. After performing a map task that collects values from the big data of
Big data analysis engine,
Stored data includes wearable sensor data, medical records, and social network content, including pre-analysis of wearable sensor data, pre-processing and filtering of wearable sensor data, pre-processing of medical records, pre-processing of social network content and web page content After performing the feature opinion word polarity using SWN (SentiWordNet), the result of feature polarity is accumulated to find the polarity of the entire document, word embedding and ontology-based feature extraction are performed,
Pre-processing of medical records uses information about drugs contained in drug data prescribed for diagnosis and some of the characteristics of medical records to classify patients. For data analysis, each attribute of the medical record is assigned an ID and a reference value representing the patient's health status, and the missing sensor-based generated data values are collected using medical record data to overcome the limitations of sensor-based generated data. to be replaced with the current medical record attribute value of
Healthcare monitoring device. delete According to claim 1,
Big data analysis engine,
To provide additional information about the neural network model that understands the meaning of words, perform ontology-based feature extraction to retrieve all ontology information, integrate with word embeddings for classification, and extract information from the ontology, term frequency ( defined mathematically using statistical methods including Term Frequency (TF), Term Frequency, and TF-IDF (Inverse Document Frequency).
Healthcare monitoring device. According to claim 1,
Big data analysis engine,
Using the information gain (IG) method calculated through Waikato Environment for Knowledge Analysis (WEKA), we extract the features of the wearable sensor data, find the value of the attribute, and use the entropy to measure the uncertainty and the previous entropy to define the entropy difference between
Healthcare monitoring device. Collecting physiological data from a wearable device through a data collection unit, drug and symptom data from a smart terminal, and discussion data of patients and doctors on social networks through application programming interfaces (APIs);
The data storage unit is connected to the personal cloud server to store the collected physiological data, drug and symptom data, and discussion data; and
The stored data is analyzed through big data analysis engine to identify feature polarity, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and calculation of data including drug side effects prediction and performing classification
including,
The data storage unit is connected to the personal cloud server to store the collected physiological data, drug and symptom data, and discussion data,
It receives wearable sensor data, social network content, and web page content from a personal cloud server and transmits it to HDFS (Hadoop Distributed File System) through Flume. It is a distributed file storage system that provides bandwidth exceeding that of HDFS for transfer, uses Apache Pig to transfer wearable sensor data, social network content, and web page content to MapReduce, and paired key values through MapReduce. After performing a map task that collects values from the big data of
The stored data is analyzed through big data analysis engine to identify feature polarity, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and calculation of data including drug side effects prediction and performing the classification,
Stored data includes wearable sensor data, medical records, and social network content, including pre-analysis of wearable sensor data, pre-processing and filtering of wearable sensor data, pre-processing of medical records, pre-processing of social network content and web page content After performing the feature opinion word polarity using SWN (SentiWordNet), the result of feature polarity is accumulated to find the polarity of the entire document, word embedding and ontology-based feature extraction are performed,
Pre-processing of medical records uses information about drugs contained in drug data prescribed for diagnosis and some of the characteristics of medical records to classify patients. For data analysis, each attribute of the medical record is assigned an ID and a reference value representing the patient's health status, and the missing sensor-based generated data values are collected using medical record data to overcome the limitations of sensor-based generated data. to be replaced with the current medical record attribute value of
How to monitor healthcare. delete 6. The method of claim 5,
The stored data is analyzed through big data analysis engine to identify feature polarity, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and calculation of data including drug side effects prediction and performing the classification,
To provide additional information about the neural network model that understands the meaning of words, perform ontology-based feature extraction to retrieve all ontology information, integrate with word embeddings for classification, and extract information from the ontology, term frequency ( defined mathematically using statistical methods including Term Frequency (TF), Term Frequency, and TF-IDF (Inverse Document Frequency).
How to monitor healthcare. 6. The method of claim 5,
The stored data is analyzed through big data analysis engine to identify feature polarity, word embedding and ontology-based feature extraction, extraction from wearable sensor data, data component analysis, Bi-LSTM-based diabetes and BP classification, and calculation of data including drug side effects prediction and performing the classification,
Using the information gain (IG) method calculated through Waikato Environment for Knowledge Analysis (WEKA), we extract the features of the wearable sensor data, find the value of the attribute, and use the entropy to measure the uncertainty and the previous entropy to define the entropy difference between
How to monitor healthcare.

Images