KR20210004057A - Machine Learning and Semantic Knowledge-based Big Data Analysis: A Novel Healthcare Monitoring Method and Apparatus Using Wearable Sensors and Social Networking Data - Google Patents

Abstract

The present invention provides a novel medical monitoring method using a wearable sensor and social networking data by big data analysis based on semantic knowledge and machine learning and an apparatus thereof to improve accuracy of classification. According to the present invention, the novel medical monitoring apparatus using a wearable sensor and social networking data by big data analysis based on semantic knowledge and machine learning comprises: a data collection unit to collect data collected by a wearable device or a sensor, medical records of a patient, discussion data on social networks of patients and doctors, and data of a medical webpage; a data storage unit of a big data cloud server connected to individual cloud servers to store collected data; and a big data analysis engine to perform data analysis including pre-analysis of collected data, preprocessing and filtering of collected sensor data, preprocessing of medical records, and preprocessing of social network content, and perform feature polarity identification and document labeling. The big data analysis engine extracts word embedding and ontology-based features in text data of stored data, extracts properties from the wearable device or sensor data, performs a statistical approach for reducing dimension through principal component analysis, and predicts Bi-LSTM-based diabetes, BP classification, and medication side effects.

Description

Machine Learning and Semantic Knowledge-based Big Data Analysis: A Novel Healthcare Monitoring Method and Apparatus Using Wearable Sensors and Social Networking Data}

The present invention relates to a new medical monitoring method and apparatus using wearable sensors and social networking data through machine learning and semantic knowledge-based big data analysis.

With advances in information technology, all tools and devices in the medical industry have become digital. This digitization has created a strong relationship between patient and technology. The main purpose of advanced devices is to be easily used for patient medical monitoring. In addition, various devices such as smartphones and wearable sensors are used in daily life. Smartphones may include sensors that can be used to obtain human body information. Wearable devices can be used to collect vast amounts of physiological patient information. However, due to social change, the human need for a healthy life is increasing. In addition, health-related problems have shifted from serious to chronic. Therefore, extracting valuable information from devices and analyzing data effectively has become a new challenge for medical monitoring systems for patients with diabetes and blood pressure (BP). On the other hand, data generated by these devices is difficult to process because they are not structured. In addition, the amount of data is growing rapidly, requiring huge storage space. Therefore, there is a need for a smart methodology and cloud-based medical architecture to accurately monitor diabetes and BP patients, store medical data, and perform predictive analysis on structured and unstructured data.

Various factors such as emotional state and social stress can affect the patient's health. Recently, the use of social networking in the medical industry is increasing rapidly. People with diabetes and abnormal BP share their feelings and experiences on Social Network Services (SNS). They share valuable information and motivate each other to fight diabetes and high and low BP. In addition, diabetic patients publish their opinions on specific drugs. New patients see other people's opinions and react to the same medication. Therefore, the health management monitoring system for diabetes or abnormal BP needs social networking data to identify patients' emotional disorders using posts and monitor drug side effects using drug reviews. However, the information on SNS about the patient's emotion and drug experience is unstructured and unexpected information, and it will be a difficult task for the medical monitoring system to extract and analyze the information to monitor the patient's mental health and predict drug side effects.

Machine Learning (ML) technologies such as Decision Trees, Support Vector Machines (SVMs), K-Nearest Neighbors (KNNs), fuzzy logic, and Multilayer Perceptrons (MLPs) have been used in recent years. Use to monitor diabetic and BP patients and provide appropriate treatment, but continuous patient monitoring generates large amounts of medical data such as sensor data, patient profiles, medical records, laboratory tests and doctor notes, both medical and social networking data. It has increased significantly over the years, and this is called big data (both unstructured and structured data) Traditional approaches and ML techniques may not be able to handle such data well for extracting meaningful information and making abnormal predictions. Also, such data may not be helpful to the healthcare industry until it is intelligently processed in real time, which requires a big data cloud platform and advanced deep learning methods such as Long Short Term Memory (LSTM).

New developments in medical monitoring systems are still a huge challenge as they require huge multi-disciplinary steps. With new technologies and social changes in the medical field, traditional systems are not sufficiently effective in these new conditions. A new framework based on new methods is needed to solve the problem. However, traditional techniques can be used with new systems.

Current medical industry technology plays a key role in providing an efficient way to collect physiological information about individual patients. In addition, efficient medical monitoring can be performed by collecting patient data using smartphones and wearable sensors. People also use social networking to share feelings and opinions about certain drugs, which are useful for observing emotional disorders and predicting drug side effects. However, continuous patient monitoring generates large amounts of medical data, and medical data generated by users of social networking sites is available in large quantities, unstructured, and can be unexpected. Thus, the current trend requires advanced approaches capable of processing vast amounts of medical data extracted from various sources, including smartphones, wearable sensors, medical records, and social networks (i.e., big data). Current medical monitoring systems are not efficient in extracting important information from devices and social networking data, and are having difficulty in analyzing them effectively. Also, existing machine learning techniques cannot handle and process medical data extracted for abnormal prediction. Therefore, we propose a new healthcare monitoring architecture based on a cloud environment and big data analysis engine to accurately store and analyze medical data and improve classification accuracy.

An object of the present invention is to provide a new healthcare monitoring method and apparatus based on a cloud environment and a big data analysis engine in order to accurately store and analyze medical data and improve classification accuracy.

In one aspect, a new medical monitoring device using wearable sensors and social networking data with machine learning and semantic knowledge-based big data analysis proposed in the present invention includes data collected through wearable devices or sensors, patient medical records, and patients. A data collection unit that collects discussion data on social networks of doctors and doctors, and data of medical web pages, a data storage unit of a big data cloud server that stores collected data connected to a personal cloud server, and a dictionary of collected data It includes a big data analysis engine that performs data analysis including analysis, pre-processing and filtering of collected sensor data, pre-processing of medical records, and pre-processing of social network content, and performing feature polarity identification and document labeling. The big data analysis engine extracts word embedding and ontology-based features from text data of stored data, extracts features from wearable devices or sensor data, performs a statistical approach to dimension reduction through principal component analysis, and is based on Bi-LSTM. Diabetes and BP classification and drug side effects are predicted.

The data collection unit collects the patient's medical records to determine the treatment and medical history, extracts the patient's contents from SNS to determine the patient's feelings, emotions, and stress, and uses the medical web to identify side effects of current drug consumption. The page collects patient reviews for drugs.

The big data analysis engine converts the collected data into a CSV file so that it can be parsed for preliminary analysis, displays it in the form of a column with numeric values, and then displays the ID of the sensor data as the actual sensor name, and discrepancy and noise. Filter data to remove data, preprocess different medical records, including laboratory tests, self-test answers, and drug data taken, stop word removal, tokenization, PoS tagging, and feature conversion for social network content. Converts the data into a structured form by performing pre-processing including.

The big data analysis engine uses the discussion data on the social networks of patients and doctors to detect the patient's stress and depression through a sentiment analysis approach, and to assess the patient's drugs to understand the effectiveness and side effects of diabetes treatment. To conduct a review.

The big data analysis engine applies a word embedding approach to represent a word as a numeric value, sets a dimension to reduce the dimension, expresses the word using the set dimension value, and creates a word embedding model based on a neural network of Word2vec. To express words.

In another aspect, a new medical monitoring method using wearable sensors and social networking data as machine learning and semantic knowledge-based big data analysis proposed in the present invention is a data collected through a wearable device or sensor, and a patient's medical record. , Collecting discussion data on social networks of patients and doctors, and data of medical web pages, storing the collected data by connecting to a personal cloud server through the data storage unit of the big data cloud server, and analyzing big data It performs data analysis including pre-analysis of data collected through the engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and identifying feature polarity and document labeling. Including steps, performing data analysis including pre-analysis of data collected through the big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and features The step of performing polarity identification and document labeling includes extracting word embedding and ontology-based features from text data of stored data, extracting features from wearable devices or sensor data, and performing a statistical approach to dimension reduction through principal component analysis. And predicts Bi-LSTM-based diabetes and BP classification and drug side effects.

According to embodiments of the present invention, through the proposed big data analysis engine, based on data mining technology, ontology, and bidirectional long-term memory (Bi-LSTM), medical data is efficiently pre-processed and useful functions are provided through data mining technology. It can extract and reduce the number of dimensions of the data. In addition, the proposed ontology provides semantic knowledge of the substance and aspect and the relationship in the diabetes and blood pressure domains, and Bi-LSTM can accurately classify medical data to predict drug side effects and abnormal states of patients. In addition, the proposed device is utilized for patient classification using medical data related to diabetes, BP, mental health and drug review, accurately processing disparate data through the proposed model and improving the accuracy of classification of health status and prediction of drug side effects. I can make it.

1 is a diagram illustrating a medical monitoring apparatus using machine learning and semantic knowledge-based big data analysis according to an embodiment of the present invention.
2 is a view for explaining a big data cloud server and HDFS according to an embodiment of the present invention.
3 is a diagram illustrating a big data analysis engine according to an embodiment of the present invention.
4 is a diagram for explaining embedding and ontology-based feature extraction according to an embodiment of the present invention.
5 is a diagram for describing LSTM-based classification and prediction according to an embodiment of the present invention.
6 is a flowchart illustrating a medical monitoring method using machine learning and semantic knowledge-based big data analysis according to an embodiment of the present invention.

In order to improve classification accuracy, the present invention proposes an advanced medical monitoring architecture based on a bi-directional long-term storage device (Bi-LSTM) for accurately analyzing medical big data. The proposed architecture integrates different sources of information for efficient medical monitoring of chronic patients. The proposed system was applied to the classification of patients using health data related to diabetes, BP, mental health and drug review. The results prove that the proposed model correctly handles disparate data and improves the accuracy of patient health status classification.

According to an embodiment of the present invention, a new framework is built to extract the most useful medical data in bulk from various sources such as smartphones, wearable sensors, medical records and social networks. It utilizes big data cloud storage to store extracted data, and applies MapReduce to intelligently process structured and unstructured data.

A big data analysis engine is proposed for actual big data analysis. It is used to accurately process medical data containing inconsistencies and with missing values, noise, different formats, large sizes and high dimensionality. It is also used to improve data processing quality and save time. Big data analysis engines use artificial intelligence approaches to extract useful functions and reduce the number of dimensions of data.

A neural network-based word embedding model called Word2vec is used to display healthcare text data with semantic meaning. In addition, a specific domain ontology is integrated with the Word2vec model. This ontology provides additional information about the neural network model that understands the semantic meaning of unusual words.

New semantic knowledge using Bi-LSTM models is used for classification of unstructured and structured medical data. The proposed model is based on semantic knowledge and Bi-LSTM with different classifiers (e.g., convolutional neural network (CNN), MLP, SVM, fuzzy classifier, fuzzy logic, logistic regression). , Compared to random forest, KNNN). In addition, Principal Component Analysis (PCA) and Information Gain (IG) are used together with the proposed model, and the results are compared. This comparison helps to determine the strengths and limitations of the proposed approach and classification model. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Machine learning technology and big data play a key role in chronic patient medical monitoring systems. With the rapid increase in the use of wearable sensors and social networking data in the medical field,

In ML access and medical monitoring of diabetic and BP patients using wearable sensors, there have been various studies on extraction of physiological information based on wearable sensors and health monitoring of patients with chronic diseases. These studies have introduced a variety of methods to analyze wearable sensor data. Wearable sensors and ML-based customized medical monitoring systems were provided for diabetic patient monitoring. It is a system that collects data using a Bluetooth low energy sensor. Afterwards, multilayer receptors and LSTM were applied to classify diabetes types and predict glucose levels of input users. For diabetes prediction, a framework based on ML method and Hadoop environment was proposed. In this study, an information gain algorithm for feature extraction was proposed. They also predicted diabetes using naive Bayes, decision trees, and random forests. However, as the amount of data increased, the accuracy of these classifiers decreased significantly. A 5G smart diabetes system based on wearable sensors, big data and ML was provided for customized diabetes diagnosis. These systems have five main goals (smart, personalization, comfort, effectiveness, and sustainability) to help patients detect diabetes early and provide personalized treatment solutions. A wearable technology and Internet of Things (IoT)-based recommendation system was proposed for proactive medical monitoring. These systems solve two major problems: identification of factors that should be monitored and identification of wearable technologies that must be used to measure factors. In order to effectively mine sensor data and laboratory tests, an LSTM-based model was presented. Here, LSTM and MLP techniques were trained using different attributes of intensive care unit (ICU) patients, including body temperature, systolic blood pressure, blood sugar, and heart rate. Then, the effectiveness of these classifiers was verified and their performance was compared.

A big data framework has been proposed for diabetes. These studies have provided solutions for managing diabetes and the patient's current diabetes status. In addition, the challenges of big data in the medical field were also discussed. A bidirectional LSTM that predicts blood glucose levels in the future has been proposed. In this system, the results of a simple LSTM and Bi-LSTM were compared using 26 data sets of 20 real patients. Frameworks covering various topics such as data collection, data processing, and system response were discussed, and the differences between wearable devices and human factors were also examined.

A knowledge discovery approach has been proposed for medical support. The system has simplified the analysis of big data in the cloud environment and described a new method of determining the classification to predict an abnormal condition in blood pressure patients.

In medical monitoring systems based on social networking data and ML approaches, the widely discussed medical monitoring systems utilize social networking data and drug information. An emotional health management system has been proposed to detect a patient's psychological disorder. They used patient messages posted on social networking platforms and checked their depression and stress levels. They applied CNN, Recurrent Neural Networks (RNN), and Bi-LSTM to detect stress and depression. In addition, an ontology-based recommendation system for sending text messages to patients based on the monitoring results was proposed. A deep learning-based emotional analysis for congenital depression was presented. WeChat friends-circle data was used for LSTM training, and emoticons were used as feature extraction to observe the patient's congenital depression. This system shortened examination time and reduced the cost of communication between doctors and patients. An artificial intelligence approach has been proposed to analyze patient posts on medical social networking platforms and to identify serious patient problems. The system applied pre-processing of text and ML classifiers to automatically analyze patient posts and inform doctors when needed. A new system has been proposed to identify diabetes risk based on Twitter's social networking activities. Here, a new approach to data generation was presented, and an ML method was proposed to classify users' diabetes risk based on their Twitter activity. Bi-LSTM and gated repeat units were used to intelligently process social media comments from patients on treatment. Focusing on the patient's words about drugs, he discovered medical concepts related to diseases from them. The system considered different types of text to detect the disease (for example, a depressive disorder was found in patients using text such as "Wake up too early").

A new system has been proposed to prevent adverse drug reactions (ADEs). In this system, the focus is on ADE prescriptions to improve patient safety and optimize medical procedures. A feature-based sentiment analysis of drug review was presented to detect drug side effects. The system performs different tasks for drug reviews that identify drug effects and side effects. A system based on emotional function has been proposed to detect the posts of adverse drug reactions on social networks. A large amount of text and experimentation has been conducted to develop a practical approach to identifying posts related to drug side effects.

In the field of diabetes treatment, an ontology-based emotional analysis system with a characteristic level has been proposed. Ontology was used to provide a semantic relationship between the features that were successfully classified. An ontology mapping framework was proposed to discover the relationship between the ontology features and the features required in the text. In this system, various ontology of bio portals were used, and a new approach was proposed to extract features from the ontology for word embedding. Deep learning for embedding medical knowledge has been proposed. In this work, a biomedical ontology related to epilepsy was employed to automatically find medical concepts in clinical documents. Ontology was proposed to improve the outcome of in-depth learning about disease name extraction from Twitter text. The structure of a neural network was presented using an ontology to automatically extract disease names from tweets. A system called BO-LSTM has been proposed to detect biomedical information in text. This approach overcomes the problem of missing semantic entities in word embedding. Here, to reinforce the extraction of biomedical relationships using deep learning LSTM, a domain ontology with word embedding was used.

In the medical big data and deep learning approach, the analysis and expression of medical big data with low dimensions is another major issue of medical monitoring systems. Recently, various systems have been proposed for analyzing big data, reducing the dimension of big data, and managing large amounts of different types of data using ML. In the prior art, a telemedicine application was used to collect patient data and then a fuzzy rule classifier was applied to the data. They discussed various issues related to information clustering, information retrieval, and big data parallel processing in a cloud environment. In another prior art, a recommendation system, a numerical reputation system, and a context survey have been proposed to ensure the integrity of big data. In addition, ML methods and analysis algorithms were used to process data volumes and manage data rates. In another prior art, a disease diagnosis medical framework based on cloud-centric IoT has been proposed to predict student diseases. In this system, student healthcare data was generated using medical sensors and the University of California Irvine (UCI) data set, and the ML algorithm was trained to predict student disease. In another prior art, a mobile healthcare application based on cloud and IoT has been proposed to monitor and treat serious diseases. A fuzzy neural classifier was applied for the diagnosis of severe diabetes using the UCI repository data set. In another prior art, big data analysis for an effective health recommendation system has been proposed. The system processes large amounts of structured and unstructured data extracted from patients' social networking activities. In addition, the ML algorithm was applied to recommend central treatment for patients.

In another prior art, the effect of IoT on the analysis result of the big data psychological index was discussed. Initially, the concept, characteristics, and decision value of big data sentiment analysis were introduced, and then the framework required to process big data was described. In another prior art, a random forest and map reduction were used to present big data, an analysis-based IoT healthcare system. The system considered patients with different diseases for analysis. In addition, an improved dragonfly algorithm was applied to select and classify the optimal characteristics. In another prior art, a new IoT-based framework has been proposed to collect sensor data for medical applications. This system uses Apache HBase and Apache Pig to collect and store sensor data, and predicts heart disease by applying a predictive model of map reduction. In another prior art, Twitter data was analyzed in parallel using the Hadoop environment, and Twitter data was classified into three classes: positive, neutral, and negative. The system provides a semantic similarity formula that effectively identifies the semantics between words in a tweet. Another prior art proposed a maximum entropy (MaxEnt) classifier for big data processing and medical systems based on the Hadoop framework. They used patient Twitter data and applied an emotional analysis approach to predict the patient's health status.

Most of the systems described above were based on traditional methods and dealt with the problem of big data to some extent. However, there is no clear framework for processing and analyzing big data with high accuracy using big data analytics. Also, these systems are still not enough to handle many types of structured and unstructured data. Because of the limitations of medical semantic knowledge for ML classifiers, these systems can misclassify social networking data.

1 is a diagram illustrating a medical monitoring apparatus using machine learning and semantic knowledge-based big data analysis according to an embodiment of the present invention.

The proposed medical monitoring device using machine learning and semantic knowledge-based big data analysis includes a data source 110 including a sensor 111, a hospital medical record 112, a hospital SNS 113 and a patient SNS 114, A data collection unit 120, a data storage unit 130, a big data analysis engine 140, and a data display unit 150 are included.

The data source 110 processes data of different types. Sensor devices, medical records and social networking platforms are the main sources of data.

The data collection unit 120 serves to collect data in different areas for patients with diabetes and BP. The data collection unit 120 collects physiological information from a social network discussion between a patient and a doctor through wearable devices, drug and symptom information of a smartphone, and application programming interfaces (APIs).

The data storage unit 130 offloads monitoring data of a patient using a wearable device and data collected from a social network to a cloud server through a wireless communication network.

The big data analysis engine 140 is the most important component of the proposed framework. The big data analysis engine 140 is divided into two lower layers: a data calculation unit 141 and a data classification unit 142. The data calculation unit 141 includes sub-modules such as data pre-processing 141a, data pre-analysis 141b, feature extraction 141c, dimension reduction 141d, and word embedding 141e. Here, the ontology-based semantic knowledge 143 is used to process and analyze data required for extraction of necessary information, along with a soft computing approach. The data classification unit 142 utilizes the Bi-LSTM 142a with the ontology for classification of diabetes, BP, mental health, and drug side effects. The data classification unit 142 intelligently analyzes multidimensional big data related to diabetes and BP to obtain insight into decision making from the data, and provides a customized diabetes and BP medical system to patients.

The proposed system uses Hadoop MapReduce with ML to reduce large-scale data on patient care. The final layer is the data display unit 150 that provides the analysis result to the doctor. The data display unit 150 combines the results generated for healthcare monitoring 151 and recommendation 152 with the doctor's proposed treatment through healthcare monitoring 151 and recommendation 152, and then personal diabetes and BP health. Management therapy is recommended to the patient. The proposed system helps to warn patients with diabetes and BP before their health risks reach high levels. It helps doctors provide practical treatments for their patients by observing their health status wisely.

More specifically, the data collection unit 120 according to an embodiment of the present invention collects data from four different sources. First, using smart sensors and wearable devices such as blood pressure monitors, glucometer sensors, smart watches, pulse oximeters, accelerometers, temperature sensors, weight scales, etc. to determine blood pressure, blood sugar levels, heart rate, stress ratio, oxygen saturation, temperature and weight. In order to monitor the physiological signs of the same patient, real-time body signals of the patient can be collected. In the proposed system, a wearable device is placed on the patient's body to monitor body functions. In addition, the use of mobile devices is greatly increased, and in recent years, health-related applications are well developed. Mobile devices contain sensors that provide an opportunity to collect accurate information about different parts of the body. Therefore, smartphones are used to collect diet, exercise, and other activity information from diabetics or BP patients, and to collect personal information (age, gender, height, and other information). This information is important for health care and disease prevention. However, smartphone data is highly insecure and can be easily damaged. Therefore, it is difficult to use smartphone data for accurate health management and related information extraction. In order to manage sensor data and extract related information for efficient medical service, data mining technology and ontology-based semantic knowledge were used.

Medical records are data on the treatment experienced by patients with diabetes and BP. Patient medical records are collected, including patient records (eg, treatment, laboratory tests, and drug intake). These records can be analyzed to extract valuable information that can help improve medical guidelines for diabetic and BP diagnosis. However, the volume of medical records is usually large, and each record consists of data with scattered variables and high dimensionality. In addition, patients with diabetes and BP may face other complications such as kidney and cardiovascular disease, neurosis, and skin and eye diseases. Therefore, it is necessary to analyze medical records in order to identify patients affected by the complications and monitor the condition with more specific tests.

The proposed system first extracts patient content from the hospital social networking platform. However, this task requires additional work and completely relies on social network privacy settings. APIs of some specific social networks are not disclosed. In this situation, special software such as wrappers can be used to extract the information (eg patient posts). In general, people with diabetes and BP contact their doctor on a regular basis, but they also need support, knowledge and skills for personal monitoring of their health. Also, if patients don't get efficient information from their doctors, social media can play an important role in meeting their needs. Thus, social networking platforms such as Facebook and Twitter provide opportunities for patients to gain sufficient knowledge about diabetes and BP and connect with people with similar health problems or experiences. Social networks provide a platform for sharing knowledge about diabetes treatment to both patients and doctors. It collects social media data such as drug reviews and emotional posts from patients to predict the level of stress and depression, identifies side effects of diabetic drugs in the context of diet and lifestyle, and improves patient treatment and knowledge.

The popularity of social networks has increased in the healthcare field. Thus, patients connect with online communities to meet their needs. Facebook's diverse community, including the American Diabetes Association, Diabetes Recipes and Food Hints, Diabetes Daily Newspaper, and Diabetes Health, provides a platform for patients to share information with each other. Patients can respond to posts and share with others. Data was extracted from a Facebook page using the Graph API with a Java client, RestFB. The Graph API allows you to automatically extract information from Facebook. First, select a community page that contains information about diabetics and medications. And all posts were extracted from community pages published between January 2017 and January 2019. The reactions (or reactions and emotions) made in these posts were collected and stored for further processing.

Twitter's Streaming API and REST API were used to retrieve tweets containing diabetes data. I used a REST API with various queries to retrieve the latest tweets. In these questions, the most specific terms related to diabetes should be used to retrieve the most relevant tweets. Therefore, queries based on keywords were written along with Boolean operators AND and OR, such as diabetes patient AND (blood pressure OR blood sugar), hypertension patient AND (heart rate OR stress ratio OR blood sugar), and diabetes treatment patient AND (diabetes OR drug). More than 30 keywords were used to construct a diabetes-related query. Through the aforementioned keyword-based query, 300,000 tweets were obtained about diabetics, treatments, drugs, and symptoms.

Diabetes patients are interested in knowing the side effects, diseases, and symptoms of taking certain medications. Usually patients share their experiences of drug side effects on the same social media website. These websites contain a huge amount of information related to the side effects of antidiabetic drugs. Therefore, these websites were chosen as the data sources for the proposed system. Here, the name of the drug for diabetes and BP was used as a lookup, and 1600 posts were searched, and the posts containing only drug-related information were extracted using a keyword-based search engine. In addition, the proposed system employs an automatic web crawler that collects 25,000 user comments (reviews) on drugs.

2 is a view for explaining a big data cloud server and HDFS according to an embodiment of the present invention.

The data storage unit 130 of the big data cloud server considers four different data sources to monitor the patient's condition and provide useful information. Here, the wearable device data is used to extract physiological information about individual patients. Second, the patient's medical records are collected to determine treatment and medical history. Third, the patient's content is extracted from SNS to understand the patient's feelings, emotions, and stress. Fourth, patient reviews of drugs are collected on medical web pages to identify side effects of current drug consumption. These large amounts of data from four different sources are difficult to store and process. In addition, the number of patients suffering from various diseases is increasing rapidly, and as a result, a lot of data is being generated in the medical industry. Therefore, we need a smart approach to manipulating and processing data skillfully. Therefore, it utilizes big data cloud storage so that the extracted data can be easily accessed anytime, anywhere. Data must be stored intelligently so that information can be retrieved accurately and quickly. The data used in the proposed work is contained in large volumes with versatility and speed. Therefore, the existing storage and retrieval methods may not process data from different sources.

The proposed system connects to a private cloud server called Amazon S3, and is highly scalable and secure for storing patient information. Amazon S3 stores data in buckets and processes them for other purposes. Each bucket is assigned a unique name and URL, and data is uploaded to Amazon S3 using the s3cmd method. The s3cmd method allows users to upload, search, and manage data in a cloud database.

Amazon S3 provides the ability to upload data to the HBase (231) cluster. As shown in FIG. 2, the wearable sensor and social networking data are transmitted from the cloud 210 server to the Hadoop Distributed File System (HDFS) 232. HDFS (232) is a distributed file storage system that provides a high bandwidth to transmit data to the HBase (231) cluster. HBase 231 runs on top of HDFS 232. It is a distributed database management that stores data in the form of rows and columns. Flume 220 is used to transfer data from the cloud 210 server to the Hadoop ecosystem. Flume 220 includes three units (source, decorator, sink), and the source of data, decoration of data (eg, compression and decompression), and target of data for a specific purpose are presented, respectively. Apache Pig 241 is used to transmit data extracted by the wearable device to MapReduce 242. Apache Pig 241 processes structured and unstructured data and presents it for data analysis. MapReduce 242 is Hadoop's parallel processing system for large datasets operating in a distributed environment. It has two main functions. That is, Map and Reduce. The map task collects values from big data of paired key values. The reduction function saves the value set for a specific key. Map reduction intelligently processes structured and unstructured data. Thereafter, the stored data is transmitted to the analytics engine 250.

3 is a diagram illustrating a big data analysis engine according to an embodiment of the present invention.

The data consists of sensory data, medical records and social networking data. However, it is extremely difficult to process real big data because of inconsistencies, missing values, noise, different formats, large size, and high dimensionality. Low quality and noisy data leads to poor quality results. Apply data processing steps before actual processing to improve data processing quality and save time. As shown in FIG. 3, the proposed system includes pre-analysis of sensor data, pre-processing and filtering of sensor data, pre-processing of medical records, and pre-processing of social network content.

In the pre-analysis of sensor data, patient physiological information is extracted using wearable biomedical and behavioral sensors. As shown in Table 1, different parameters are detected from diabetic and BP patients using sensors and smartphones.

resource ID parameter Explanation



sensor S1 Blood sugar Blood sugar (mg/dL) S2 temperature Patient's current body temperature S3 Blood pressure Systolic blood pressure (mmHg) Diastolic blood pressure (mmHg) S4 Oxygen saturation SpO ₂ consumption by patient (mmHg) S5 Heart rate Patient's heart rate (bpm) S6 ECG Patient's electrocardiogram using ECG sensor S7 EEG Patient EEG test using EEG sensor S8 stress Calculation of patient stress using ECG + EEG pattern


Smartphone SP1 age Date of birth patient's age SP2 key An optometrist to find the patient's height SP3 BMI Body mass index (kg/m ² ) SP4 gender Patient's gender (0/1) SP6 activity Lifestyle: sedentary, lightly active, moderately active, active or very active







Hospital medical records MR1 Lipoprotein level Low-density lipoprotein levels (LDL cholesterol) High-density lipoprotein levels (HDL cholesterol) MR 2 hemoglobin Patient's glycohemoglobin (A1c) (%) MR 3 Blood sugar Patient's blood test MR 4 Serum creatinine Patient's blood test MR5 Triglycerides Patient's blood test MR6 cholesterol Patient's blood test MR7 AST (SGOT) Blood tests to check for liver damage MR8 ALT (SGPT) Blood tests to check for liver damage MR9 Taking drugs Extracted from the prescription list MR10 smoking Yes / No (self test) MR11 Drinking Yes / No (self test) MR12 Indigestion Yes / No (self test) MR13 Family medical history Yes / No (self test) MR14 Patient's medical history Yes / No (self test)

The parameters detected include diabetes, abnormal BP, and most symptoms of other diseases. It also extracts other parameters from the patient's body. However, only these parameters are mentioned in the proposed work for simplicity. Pre-analysis performs various steps. First, the data set is converted into a comma-separated values (CSV) file for easy parsing. The name is assigned to each attribute (parameter) and is displayed as a column with a numeric value. Then, the ID of the sensor data is displayed as the actual sensor name (for example, S1 is the blood glucose ID). Different properties are used for different purposes. For example, blood sugar, blood pressure, heart rate, body mass index (BMI), age, sex, and activity attributes are used to classify the health status of diabetic and BP patients. However, instead of using explicit years and hours for age and activity respectively, the attribute age is divided into days, months, and years, and activity hours are divided into morning, afternoon, and evening. These generated features provide valuable insights into patient health, such as patterns of normal and abnormal conditions, depending on whether a serious condition occurs during the start or end of the month or the start and end months. The final data set is then uploaded to the Hadoop cloud environment for further processing.

Data collected using wearable sensors has various limitations. They contain a lot of inaccurate and useless information. In addition, sensor data is damaged by signal artifacts such as noise and missing values, which significantly degrades classification performance. Therefore, in the pre-processing and filtering of sensor data, the data is pre-processed and filtered before analysis. Filter the data to remove inconsistencies and noise. Clean up the data by removing ASCII characters. To remove such noise from the data, we use a well-known filtering method called Kalman filtering. In addition, an unsupervised filter called ReplaceMissingValues is used to replace all missing numeric values in the data set with the means and modes of available data. Unattended features are removed with a variance of up to 90% using an unsupervised filter called the RemoveUseless filter. The numeric values are then normalized using a normalization filter and limited to between 0 and 1 for all classifications. After these steps, EmEditor is used to divide the data set into n data files and upload it to the Hadoop cloud environment for further processing.

In the pre-processing of the medical record (MR), the data in the MR consists of the entire patient record in digital format. This data contains different medical data describing the patient's health, such as laboratory tests, self-test answers, and medications taken. Laboratory tests are data from medical devices that can be used to judge a patient's health status in terms of a reference value. Also, the evaluation of the patient's health status may vary depending on the patient's medical history and family medical history. Self-diagnosis data consists of data that cannot be extracted through personal medical devices such as indigestion period, and drinking and smoking habits. Drug data contains information about drugs prescribed for diagnosis. Some of the characteristics of MR can be used for patient classification. Therefore, ID and reference value (0, 1, 2) are assigned to each attribute of MR for data analysis. Reference values 0, 1, and 2 represent the patient's health status (normal, pre-diabetes, diabetes). Also, MR data is used to overcome the limitations of sensor-based generated data. For example, it replaces the missing value with the current MR attribute value from the data set.

Pre-processing of social network content is an important task before text classification. It reduces noise arising from corpus (corpus) data and transforms the data into useful representations for ML classifiers. The proposed system collects social network content and drug reviews and stores them in HDFS. Apply the following steps to transform the data into a structured form. In addition, these steps remove useful data, helping to extract features and opinion words easily.

Words such as prepositions (to, in, and of), all articles (a, an, and the ), symbols (#, @, etc.), and URLs of corpus data do not interfere with the meaning of the document. However, the accuracy of text classification decreases. Therefore, it is essential to remove them to reduce the noise in the text. Delete this content using a well-known method called Rainbow.

Tokenization separates the corpus complex text into small terms or tokens by removing white spaces and separators. Usually white spaces and separators occur in complex text. Therefore, the white space separator is deleted by applying an n-gram tokenizer. The output is then saved for further analysis such as speech (PoS) tagging and headwords of the extracted words.

PoS tagging defines words in text. Break the corpus text into sentences, then use Stanford Core Natural Lanquage Processing (CoreNLP) for POS tagging. After tagging, every sentence is identified as having a complete clause with a noun and a verb.

Stemming and lemmatization: Stemming transforms words in corpus text into their own basic form. This system applies the suffix dropping algorithm for stemming. Vocabulary expresses the vocabulary of words used in the text. After lexicalization, the system easily obtains vocabulary information for each word. For example, blood sugar is related to blood glucose. Thus, stems and headwords are utilized for further processing.

In feature transformation, the patient uses a peculiar word (e.g., depressssssed) in SNS that affects the classifier result. Thus, it converts a sequence of characters that appear more than once into a normal word (for example, depressssed becomes depressed).

In feature (shape) polarity identification and document labeling, the proposed system detects patient stress and depression using content posted on social networks. They also perform a number of tasks on drug review to get their opinions on the effectiveness and side effects of diabetes treatments. We use the sentiment analysis approach for the two tasks mentioned earlier. Therefore, it is important to find the feature polarity and document labeling to distinguish sentiment. After pre-processing the social network and web page content, we use SentiWordNet (SWN) to determine the polarity of the feature opinion word. Then, the result of the feature polarity is accumulated to find the polarity of the entire document. SWN is a vocabulary resource that connects each synchronization set of WordNet with three numbers: positive, negative, and target. However, SWN represents a noun, verb, adjective, or adverb with a sense of each word. Therefore, Word Sense Disambigation (WSD) is used to deal with this problem by extracting the category sense required for each word. In addition, if the SWN does not have any meaning, a value of 0 is assigned to the input word. After WSD, the SWN score for the same sense of the opinion word is extracted, and the polarity of each feature is calculated using the following equation.

(1)

(One)

(2)

(2)

(3)

(3)

Here, Pos _score SWN _w , Neg _score SWN _w , and Neu _score SWN _w represent positive, negative, and neutral scores of the feature (shape), respectively. This score is calculated by means of SWN arithmetic for each word w. If Pos _score (F _i )> Neg _score (F _i ) and Neu _score (F _i ), the system regards the feature polarity as positive. In contrast, if Neg _score (F _i )> Pos _score (F _i ) and Neu _score (F _i ), the feature polarity is negative. Finally, if Neu _score (F _i )> Pos _score (F _i ) and Neg _score (F _i ), it is considered neutral.

4 is a diagram for explaining embedding and ontology-based feature extraction according to an embodiment of the present invention.

In word embedding and ontology-based feature extraction from text data, word embedding is a word expression method in which words in a document are encoded as numeric values for advanced analysis. Two well-known approaches can be applied to word expression. One is hot word encoding and one is embedding. In this work, we apply a word embedding approach to represent words as numeric values. After setting the dimensionality, d, the word is expressed using the d-dimensional value. For example, when d is set to 3, "metformin" is expressed as {0.3, 0.6, 0.8}. This method greatly reduces the dimensions. However, the embedding approach avoids the relationship between words in the matrix. Therefore, a neural network-based word embedding model called Word2vec is used for word expression. Word2vec includes two architectures: continuous bag-of-words (CBoW) and skip-gram model. The CBoW model uses the context around the word for word prediction. The skip-gram model predicts the surrounding situation by using the current word as shown in FIG. 4.

Word2vec's skip-gram model was trained using 200-dimensional vectors in the data set. It identifies the relationship between words and helps to predict the neighboring words of the input word. This Word2vec model is used to train an ML classifier to detect associations between shapes. In general, ML classifiers miss the basic semantics of features depending on the specific domain. Ontology may represent the semantics of a given domain in some cases.

Ontology aims to provide semantic knowledge of concepts and their relationships in a specific domain. The biomedical ontology, available online, covers a variety of topics related to diabetes, depression, high blood pressure, drugs, and food. In the present invention, a specific domain ontology is used, in which each class of the ontology is a concept or characteristic of a domain, and the attribute is a relationship between characteristics. This is a basic expression of the online biomedical ontology available at the National Biomedical Ontology Bio Portal Center, a common way to formulate knowledge of current features.

Ontology and abbreviations Justice Number of classes Number of features Number of individuals Nutritional Ontology (ONS) Provides an explanation of complex nutrition research. 3442 66 104 BioMedBridges Diabetes Ontology (DIAB) Represents the relationship between diabetes phenotypes for text mining. 375 4 0 Diabetes Treatment Ontology (DMTO) It provides interoperability for diabetes treatment. 10700 315 63 Human Disease Ontology (DOID) It represents the concept of a rare disease. 12694 15 0 Drug Target Ontology (DTO) It provides information on the classification of drug target data. 10075 0 0 Type 1 Diabetes Ontology (FASTO) based on FHIR and SSN Provides insulin management information for diabetics. 9577 822 460

As shown in Table 2, Ontology for nutrition research (ONS), BioMedBridges Diabetes Ontology (DIAB), Diabetes Mellitus Treatment Ontology (DMTO), Human Disease Ontology (DOID), Drug Target Ontology (DTO), Fast Healthcare Interoperability It utilizes the latest edition of Type 1 Diabetes Ontology (FASTO) based on resources (FHIR) and semantic sensor network (SSN). This ontology provides additional information for neural network models that understand the semantic meaning of unusual words. It also affects word-level semantics in word embedding and text classification.

The system retrieves all ontology information and attempts to integrate it with the word embedding for the classifier. Typically, the Bag-of-Words (BOW) model is used for features. Therefore, we think of each ontology as a BOW. In order to extract information from the ontology, a well-known statistical method called term frequency (TF), term frequency and TF-IDF (Inverse Document Frequency) is used. Here, we consider each concept or feature of ontology as a term, and ontology as a document. Therefore, TF is a term (ambient word) found in ontology. TF is mathematically defined in the following equation.

(4)

(4)

If the TF(Term, Onto) value is greater than 0, repeat this step to extract a specific concept. TF-IDF selects exemplary terms for information extraction. IDF mainly reduces the meaning of words that appear in all ontology (eg, patient, disease, hospital, etc.). When a term occurs in more ontologies or more concepts in a single ontology, it means that it is a regular term and may not be a term necessary for information extraction. Therefore, the result of the logarithmic function of the input word will be reduced to zero. This shows that the value of TF-IDF is small compared to this term. The statistical explanation of IDF is shown in the following equation.

(5)

(5)

는 데이터베이스의 전체 온톨로지 수 또는 온톨로지 내의 총 개념 수를 표시한다(예를 들어,

및 |{Onto∈Ontos:Term ∈Onto}|는 용어가 나타나는 온톨로지 내의 온톨로지 또는 개념의 수이다). 다음 TF-IDF 수학식을 사용하여 공통 용어를 제거한다.here

Indicates the total number of ontology in the database or the total number of concepts in the ontology (for example,

And |{Onto∈Ontos:Term ∈Onto}| is the number of ontology or concepts in the ontology in which the term appears). The common term is removed using the following TF-IDF equation.

(6)

(6)

The TF-IDF result shows how essential the ontology function is to ontology in the ontology corpus or how important the ontology function is to the concept in the ontology.

5 is a diagram for describing LSTM-based classification and prediction according to an embodiment of the present invention.

A higher concept can be extracted based on the value of the TF-IDF term (X=2). Assume that the system obtains text related to diabetes as shown in FIG. 6. In FIG. 5, an ambiguous word for information extraction is in bold type (eg, hyperlipidemia). Once semantic information for these words is identified, the system extracts the specific concept from the ontology and provides additional information for word embedding and ML classifiers (eg, hyperlipidemia is a complication of disease).

In terms of the characteristics extracted from wearable sensor data, the massive size of the data is another major issue related to sensor-based medical monitoring systems. Raw data from the patient's body extracted in large quantities is a burden on data processing. It is important to reduce the size of the data without losing useful information. The data set contains many attributes for patients with diabetes and BP. However, not all attributes will be required for patient classification. Unnecessary attributes are time consuming and the accuracy of classification is poor. Various methods such as dragonfly algorithm and recursive shape removal are used for feature selection. It uses the IG method, which affects the classifier by reducing noise and irrelevant features.

The information gain selects shape based on the contribution of information related to the variables of the class without seeing the attribute interaction. Every attribute or feature of a data set has importance, and based on that importance the system can learn about a particular problem. Information gain is calculated using the Waikato Environment for Knowledge Analysis (WEKA). It includes various methods for processing data. The result is obtained by applying the information gain filter "infoGainAtrribeVal" as an evaluator for the diabetes and BP data set. However, IG cannot be used for numeric data sets. Therefore, it is important to convert numeric data into nominal data before using IG. Once the attribute value is identified, the IG measurement is associated with a reduction in entropy of the training data set. This approach finds the attribute value by calculating the IG according to the classification. The proposed IG method uses entropy to measure system uncertainty and finds the difference between the previous and subsequent entropy. It specifies the amount of additional information about A provided by B, as shown in Equation 7.

(7)

(7)

Here, A and B are separate variables. A is a feature, and the previous entropy can be measured using Equation 8.

(8)

(8)

는

의 이산 값에 대한 사전 확률을 나타낸다. A의 조건부 엔트로피는, 이후의 엔트로피 B가 주어진 후에, 수학식 9와 10에서와 같이 정의될 수 있다.here

Is

Represents the prior probability for the discrete values of. The conditional entropy of A can be defined as in Equations 9 and 10 after the entropy B is given.

(9)

(9)

(10)

(10)

The information gain can be calculated by putting Equations 8 and 10 into Equation 7 as in Equation 11.

(11)

(11)

Principal component analysis

The number of dimensions of the data set increases after pre-processing and additional attribute creation. This leads to problems such as reduction in classification accuracy, fit and time complexity. Therefore, we use principal component analysis (PCA), a statistical approach to dimensionality reduction. Convert p-dimensional X data to q-dimensional Y data with the least loss. The main purpose of PCA is to find the projection vector with the highest variance. Unique values are used to represent the vector achieved. The projection vector A in the vector B of the same plane can be illustrated using Equation 12.

(12)

(12)

및 데이터 X를 곱하여 수학식 13과 같이 감소된 데이터 세트를 구한다.After obtaining the projection vector, the transformation of the eigenvector to the eigenvalue is used to obtain the covariance from the data. Eigenvalues are arranged in descending order. Then, the matrix columns are sorted using the u eigenvectors corresponding to the arranged v eigenvalues. This approach is used to achieve the best projection provided by the W projection matrix. Achieved project matrix, W

And data X to obtain a reduced data set as shown in Equation 13.

X (13)Y = W

X (13)

Machine learning approaches such as CNN, MLP, SVM, logistic regression, decision tree, and KNN in Bi-LSTM-based diabetes and BP classification and drug side effects prediction are used for classification, feature selection, data normalization and statistical analysis. It is a useful algorithm. This approach does the job using supervised and unsupervised methods. However, if the data size is large, it takes a lot of time and decreases performance. Therefore, the proposed medical monitoring system uses Bi-LSTM to classify patients' diabetes, BP, mental health, and side effects of diabetic drugs. The recommendation system is activated by diabetes, high blood pressure, stress, depression, and severe side effects of diabetes medications. 5 shows the structure of Bi-LSTM for classification of diabetes, BP, mental health, and drug side effects. The grid search optimization algorithm is applied to identify the optimal value for the hyperparameter of the LSTM model. The optimal values chosen for the hyperparameter dropout rate, epochs, batch size and learning rate are 0.3, 30, 32 and 0.001, respectively. As shown in Figure 6, LSTMs are used to classify four different types of data sets. For diabetes classification, the Pima Indians diabetes data set was collected from the UCI machine learning repository. The Pima Indians data set contains records of 768 patients, of which 268 tested positive for diabetes while 500 were tested normally. There are 8 input attributes in the data set. However, only six attributes are used to train the diabetes classification model. These attributes are age, BMI, BP, plasma glucose, diabetic lineage function, and 2-hour serum insulin. The BP classification model was trained using the PhysioNet multi-parameter intelligent monitoring data set from the Intensive Care II (MIMIC-II) database. This data set consists of a number of samples of vital signs, including BP and heart rate (HR). Data sets for predicting drug adverse events were collected from the UCI repository. This data set consists of six attributes. However, in the proposed work, only two attributes are considered: name of drug and patient review.

), 입력 게이트(

), 포겟 게이트(

), 현재 메모리 셀(

), 출력 게이트(

)의 5가지 주요 구성요소로 구성된 RNN의 일종이다. 이러한 구성 요소는 이전 데이터의 업데이트 및 사용을 제어한다. LSTM 출력은 다음 수학식을 사용하여 계산할 수 있다.LSTM is a memory cell (

), input gate (

), Forget Gate (

), current memory cell (

), output gate (

) Is a kind of RNN composed of five major components. These components control the update and use of previous data. The LSTM output can be calculated using the following equation.

(14)

(14)

(15)

(15)

(16)

(16)

(17)

(17)

(18)

(18)

Where X ^t , w _z and w _h , and b are the input, weight matrix and bias vector of LSTM, respectively, and tanh(.) and σ(.) are the hyperbolic tangent function and S-shaped function, respectively. The final output of the LSTM can be calculated using the following equation.

(19)

(19)

는 출력 게이트와 입력 셀 상태 사이의 features-wise 곱셈이다. LSTM 출력은 확률론적 출력(0, 1, 2)을 식별하기 위해 소프트맥스 기능과 연결된다. 여기서 0, 1, 2는 환자가 각각 정상, 당뇨병 또는 당뇨병임을 보여준다. 소프트맥스 함수는 다음 수학식을 사용하여 계산할 수 있다.here

Is the features-wise multiplication between the output gate and input cell states. The LSTM output is connected to the softmax function to identify the probabilistic output (0, 1, 2). Here, 0, 1 and 2 show that the patient is normal, diabetic or diabetic, respectively. The softmax function can be calculated using the following equation.

(20)

(20)

Where c and x ^st are the input values of the feature category and time step k, respectively. After pre-processing, feature extraction, dimensionality reduction, and word embedding, a series of inputs (features and word vectors) representing patient physiological information and drug reviews are fed into the Bi-LSTM layer. We have developed four Bi-LSTM based classifier models ^: LSTM ^a , LSTM ^b , LSTM ^c , and LSTM ^d .

In the present invention, a wearable device and a smart phone are used to collect personal physiological information of a patient. The collected data is pre-processed and transformed for further processing into a structured form. Diabetes classification uses age, family, sex, activity, BMI, blood pressure, and blood sugar functions. Diabetes grade {normal, pre-diabetes, diabetes} is predicted by inputting each time step, t, and characteristic Xt in LSTM ^a unit.

The first LSTM unit of the LSTM ^a model can be predicted based on the specific characteristics of the patient category. However, all features should be used to obtain important information for classification. Thus, the order of diabetic features is assigned to the next LSTM unit, according to the results of the previous unit. This procedure is repeated with each input feature. In this way, LSTM ^a finds a valuable feature and produces an output. Its output is linked to the Softmax activation function, which predicts the diabetic category. The medical rules for classifying patients with diabetes and blood pressure are presented in Table 3.

In blood pressure and electrocardiography, wearable sensors are used to identify each patient's BP and heart rate. Patients can be evaluated from the systolic BP and diastolic BP values in millimeters of mercury, and HR per minute as shown in Table 3. However, these parameters are sometimes affected by physical activity, sleep, temperature, stress, and eating habits. Additionally, patients may have BP and HR issues due to other factors such as family profile and disease history. For example, during exercise, your HR is always high. This means that the patient's abnormal health condition is sometimes not dangerous. Therefore, it is essential to detect abnormal conditions using the patient's medical records and daily activities. Select key characteristics from medical records such as age, gender, and family history from diagnosis and treatment history. Search all diagnostic and medical records to extract information related to diabetes and BP.

The extracted information is used for feature construction. For example, if the BP data is in the patient family profile, 1 appears as a family attribute. If no data for the BP is found in the diagnostic and medical records, a null value is used for the geometry. Later, all null values are replaced with zeros. After the features were created and preprocessed, a Bi-LSTM-based model named LSTM ^b was developed, which can maintain the temporal patterns present in the sequenced data in the past.

For classification of BP, all parameters and features generated in each time step t are converted into category vectors {0, 1, 2}.

In mental health monitoring, patient messages and posts are extracted from social networks and filtered using supervised and unsupervised approaches. However, texts related to depression and stress are usually short, atypical, contain negative emotions, and have low emotional polarity. Thus, text mining and ML approaches are used to efficiently process social network content and to identify the emotional polarities of texts in terms of happiness, normal, depression or stress. Here, emotional characters are first identified, and then the text is classified by applying an emotional analysis method. The Word2vec model with ontology is used in this work to represent the text for a deep learning classifier. After training a 200-dimensional Word2vec model, the word sequence was fed into the LSTM ^c model. The output of LSTM ^c is then assigned to a softmax function, which predicts a sentiment label (eg, positive, neutral, negative or strong negative) for each sentence of the posted content. The rules for classifying a patient's mental health based on predicted polarity are shown in Table 4.

category Emotional polarity to patient posts and comments
(Positive / neutral / negative / strong voice) Emotional polarity for antidiabetic drugs
(Positive / neutral / negative)
Patient mental health classification Happiness positivity - normal neutrality - depressed voice - stress Strong voice -
Predict drug side effects No side effects - positivity Moderate side effects - neutrality Serious side effects - voice

In the prediction of adverse drug reactions, diabetes and BP drugs currently ingested are used as input queries, and reviews were searched on other websites. After filtering and pre-treatment, the reviews are automatically labeled as having no side effects/adequate side effects/severe side effects. For this labeling, identify the emotional polarity of each sentence. Then, as shown in Table 4, positive, neutral, and negative emotional polarities have no side effects, respectively, and are considered to have no moderate side effects and no serious side effects. The 200-dimensional Word2vec model was trained to represent drug reviews in a series of words in the LSTM ^d model. The output of LSTM ^d is then fed to the Softmax function, which accurately predicts drug side effects.

6 is a flowchart illustrating a medical monitoring method using machine learning and semantic knowledge-based big data analysis according to an embodiment of the present invention.

The proposed medical monitoring method using machine learning and semantic knowledge-based big data analysis includes data collected through wearable devices or sensors, medical records of patients, discussion data on social networks of patients and doctors, and medical web pages. Including the step of collecting data (610), storing the collected data by connecting to the personal cloud server through the data storage unit of the big data cloud server (620), and analyzing the collected data through the big data analysis engine. do.

In step 610, data collected through a wearable device or sensor, medical records of patients, discussion data on social networks of patients and doctors, and data of a medical web page are collected. At this time, the patient's medical records are collected to determine the treatment and medical history, the patient's content is extracted from SNS to determine the patient's feelings, emotions, and stress, and a medical web page to identify side effects of current drug consumption. Collects patient reviews for medications at

In step 620, the collected data is stored by being connected to the personal cloud server through the data storage unit of the big data cloud server.

In the step of analyzing the data collected through the big data analysis engine, the pre-analysis of data using the big data analysis engine (630), word embedding and ontology-based feature extraction from text data (640), and feature extraction from wearable sensor data (650). ), a statistical approach to dimensional reduction (660) and Bi-LSTM based diabetes and BP classification and drug action prediction (670) through principal component analysis.

At this time, data analysis including pre-analysis of data collected through the big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network content, and feature polarity identification and Do document labeling.

In more detail, the collected data is converted to a CSV file so that it can be parsed for pre-analysis, displayed in the form of a column with numeric values, and then the ID of the sensor data is displayed as the actual sensor name, so that discrepancies and noise can be resolved Filter the data to remove it. In addition, by pre-processing different medical records including laboratory tests, self-test answers, and drug data taken, and pre-processing including stop word removal, tokenization, PoS tagging, and feature conversion for social network content. Convert data into structured form.

In addition, by using the discussion data on social networks of patients and doctors, the patient's stress and depression are detected through an emotional analysis approach, and a review of the patient's drugs is conducted to obtain opinions on the effectiveness and side effects of diabetes treatment. Perform.

A word embedding approach is applied to represent a word as a numeric value, a dimension is set to reduce the dimension, and the word is expressed using the set dimension value, and the word is expressed using the word embedding model based on the neural network of Word2vec. do.

According to embodiments of the present invention, through the proposed big data analysis engine, based on data mining technology, ontology, and bidirectional long-term memory (Bi-LSTM), medical data is efficiently pre-processed and useful functions are provided through data mining technology. It can extract and reduce the number of dimensions of the data. In addition, the proposed ontology provides semantic knowledge of the substance and aspect and the relationship in the diabetes and blood pressure domains, and Bi-LSTM can accurately classify medical data to predict drug side effects and abnormal states of patients. In addition, the proposed device is utilized for patient classification using medical data related to diabetes, BP, mental health and drug review, accurately processing disparate data through the proposed model and improving the accuracy of classification of health status and prediction of drug side effects. I can make it.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It can be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (10)

A data collection unit that collects data collected through wearable devices or sensors, medical records of patients, discussion data on social networks of patients and doctors, and data of medical web pages;
A data storage unit of a big data cloud server connected to a personal cloud server to store collected data; And
Big data analysis that performs data analysis including pre-analysis of collected data, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network content, and identification of feature polarity and document labeling engine
Including,
Big data analysis engine,
Word embedding and ontology-based features are extracted from the text data of the stored data, features are extracted from wearable devices or sensor data, statistical approaches to dimensional reduction are performed through principal component analysis, and Bi-LSTM based diabetes and BP classification and Predicting drug side effects
Medical monitoring device. The method of claim 1,
The data collection unit,
To determine the treatment and medical history, collect the patient's medical records, extract the patient's content from SNS to determine the patient's feelings, emotions, and stress, and use drugs on the medical web page to identify side effects of current drug consumption. To collect patient reviews for
Medical monitoring device. The method of claim 1,
Big data analysis engine,
The collected data is converted to a CSV file so that it can be parsed for preliminary analysis, displayed in the form of a column with numeric values, and then the ID of the sensor data is displayed as the actual sensor name, and the data to remove discrepancies and noise. Filters, pre-processes different medical records, including laboratory tests, self-test answers, and drug data taken, and pre-processes including stop word removal, tokenization, PoS tagging, and feature conversion for social network content. To transform data into structured form
Medical monitoring device. The method of claim 1,
Big data analysis engine,
Using data from discussions on social networks of patients and doctors, a sentiment analysis approach is used to detect patient stress and depression, and to conduct a review of the patient's drugs to obtain opinions on the effectiveness and side effects of diabetes treatments.
Medical monitoring device. The method of claim 1,
Big data analysis engine,
A word embedding approach is applied to represent a word as a numeric value, a dimension is set to reduce the dimension, and the word is expressed using the set dimension value, and the word is expressed using the word embedding model based on the neural network of Word2vec. doing
Medical monitoring device. Collecting data collected through a wearable device or sensor, medical records of patients, discussion data on social networks of patients and doctors, and data of medical web pages;
Storing the collected data by connecting to the personal cloud server through the data storage unit of the big data cloud server; And
Perform data analysis including pre-analysis of data collected through big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and identification of feature polarity and document labeling Steps to do
Including,
Perform data analysis including pre-analysis of data collected through big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and identification of feature polarity and document labeling The steps to perform are,
Word embedding and ontology-based features are extracted from text data of stored data, features are extracted from wearable devices or sensor data, statistical approaches to dimensionality reduction are performed through principal component analysis, and Bi-LSTM-based diabetes and BP classification and Predicting drug side effects
Medical monitoring method. The method of claim 6,
Collecting data collected through wearable devices or sensors, medical records of patients, discussion data on social networks of patients and doctors, and data of medical web pages,
To determine the treatment and medical history, collect the patient's medical records, extract the patient's content from SNS to determine the patient's feelings, emotions, and stress, and use drugs on the medical web page to identify side effects of current drug consumption. To collect patient reviews for
Medical monitoring method. The method of claim 6,
Perform data analysis including pre-analysis of data collected through big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and identification of feature polarity and document labeling The steps to perform are,
The collected data is converted to a CSV file so that it can be parsed for preliminary analysis, displayed in the form of a column with numeric values, and then the ID of the sensor data is displayed as the actual sensor name, and the data to remove discrepancies and noise. Filters, pre-processes different medical records, including laboratory tests, self-test answers, and drug data taken, and pre-processes including stop word removal, tokenization, PoS tagging, and feature conversion for social network content. To transform data into structured form
Medical monitoring method. The method of claim 6,
Perform data analysis including pre-analysis of data collected through big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and identification of feature polarity and document labeling The steps to perform are,
Using data from discussions on social networks of patients and doctors, a sentiment analysis approach is used to detect patient stress and depression, and to conduct a review of the patient's drugs to obtain opinions on the effectiveness and side effects of diabetes treatments.
Medical monitoring method. The method of claim 6,
Perform data analysis including pre-analysis of data collected through big data analysis engine, pre-processing and filtering of collected sensor data, pre-processing of medical records, pre-processing of social network contents, and identification of feature polarity and document labeling The steps to perform are,
A word embedding approach is applied to represent a word as a numeric value, a dimension is set to reduce the dimension, and the word is expressed using the set dimension value, and the word is expressed using the word embedding model based on the neural network of Word2vec. doing
Medical monitoring method.

Images