KR102647385B1 - Apparatus for predicting swallowing disorder by analyzing voice patterns based on artificial intelligence and method for same - Google Patents

Abstract

The present invention relates to an apparatus and method for analyzing voice patterns using artificial intelligence and predicting whether a person has a swallowing disorder accordingly.
According to the present invention, serious complications such as aspiration pneumonia, which is the main cause of death in patients with neurological diseases such as stroke patients, can be predicted non-invasively rather than using existing invasive methods, and an artificial intelligence model using voice data and important associated factor data By performing predictions through , accuracy and reliability can be increased.

Description

Device and method for predicting swallowing disorder by analyzing voice patterns based on artificial intelligence {Apparatus for predicting swallowing disorder by analyzing voice patterns based on artificial intelligence and method for same}

The present invention relates to an apparatus and method for analyzing voice patterns using artificial intelligence and predicting whether a person has a swallowing disorder accordingly.

Stroke refers to a neurological deficit that occurs suddenly due to an abnormality in cerebral blood flow. Medically, it is called CerebroVascular Accident (CVA). In English-speaking countries, it is called stroke, and in Oriental medicine, it is usually called stroke.

The brain accounts for only 2% of the total body weight, but blood flow to the brain is as much as 15% of the total cardiac output, and oxygen consumption is as much as 20% of the entire body's oxygen consumption.

Additionally, since the brain uses only glucose as an energy source, it is easily damaged even if energy supply is interrupted even for a moment. When energy sources decrease due to a decrease in cerebral blood flow, a stroke eventually occurs.

Risk factors for stroke are very diverse, but typical examples include high blood pressure, diabetes, dyslipidemia, smoking, alcohol consumption, and atrial fibrillation. Stroke is a disease that has a high mortality rate, requires a lot of money and time to treat, and can easily leave behind disabilities even if treated. In particular, after a stroke, many patients develop swallowing disorders that make it difficult to swallow food or water.

In this way, when dysphagia occurs due to a stroke, the prognosis is extremely poor and it is not uncommon for the patient to die from pneumonia, sepsis, etc. or remain permanently disabled.

To prevent this, the stroke diagnosis and treatment guidelines recommend active diagnosis and treatment of swallowing disorders, such as checking the presence and severity of swallowing disorders and administering tube feedings if necessary.

However, there is currently no other way to check for dysphagia other than having the patient swallow water or a contrast agent and then performing radiography.

However, the problem that arises from this method is. The act of having the patient swallow water or contrast medium during the test itself can cause aspiration in the patient, increasing the risk of aspiration pneumonia and unnecessary exposure to radiation.

Therefore, there is a need for a non-invasive method of predicting the presence or absence of swallowing disorders in a simpler manner and to minimize harm to patients.

Meanwhile, Patent Document 1, which will be described later, discloses a method of determining whether a person has a stroke through voice analysis, but this only determines whether a person has a stroke and does not directly or indirectly determine whether a swallowing disorder exists. Moreover, since swallowing disorders are caused by various factors other than speech patterns, it is somewhat difficult to use them to predict whether or not there are swallowing disorders.

KR 10-1958188 B1

Accordingly, the present invention was developed to solve the above-described conventional problems, and aims to provide a non-invasive and highly accurate swallowing disorder prediction device and method.

A device that predicts swallowing disorders based on artificial intelligence, comprising: an input unit where voice data, important associated factor data, and diagnostic data of multiple stroke patients are input; a conversion unit that converts voice data input to the input unit into visualization data; a modeling unit that constructs a prediction model through an artificial intelligence model using learning data that uses the visualization data and the important correlation factor data as independent variables and the diagnostic data as a dependent variable; and a prediction unit that predicts whether or not there is a swallowing disorder by applying prediction target data including voice data and important associated factor data of the patient to be predicted to the prediction model, wherein the important associated factor data is used when measuring the voice data. Physical data including age, height, weight, waist circumference, and gender data; stroke data including type of stroke, size of stroke, severity of stroke, location of stroke, presence of small vessel disease, and cranial nerve examination findings; Underlying disease data including data on the presence or severity of high blood pressure, diabetes, hyperlipidemia, angina pectoris, myocardial infarction, metabolic syndrome, heart failure, oral disease, and vocal cord disease, and the presence or frequency of smoking and alcohol intake, and the presence, frequency, or type of drug intake. A device including lifestyle data including data is provided.

In addition, (a) inputting voice data, important associated factor data, and diagnostic data collected from multiple stroke patients; (b) converting the input voice data into visualization data; (c) constructing a prediction model by using learning data with the visualization data and the important correlation factor data as independent variables and the diagnostic data as a dependent variable by an artificial intelligence model; (d) predicting whether or not there is a swallowing disorder by applying prediction target data including voice data and important associated factor data of the patient to be predicted to the constructed prediction model; comprising: predicting whether or not there is a swallowing disorder; The important associated factor data included are physical data including age, height, weight, waist circumference, oral structure, and gender data at the time of voice data collection, stroke type, stroke size, stroke severity, and stroke Stroke data including location of occurrence data, underlying disease data including the presence or severity data of high blood pressure, diabetes, hyperlipidemia, angina pectoris, myocardial infarction, metabolic syndrome, heart failure, oral disease, and vocal cord disease, and the presence or frequency of smoking and alcohol consumption. , and provides a method of including lifestyle data including drug intake, frequency, or type data.

The artificial intelligence model is preferably a convolutional neural network (CNN).

Additionally, a recording medium recording a computer program for executing the method and a computer program recorded on the recording medium for executing the method are provided.

According to the present invention, in order to predict serious complications such as aspiration pneumonia, which is the main cause of death in patients with neurological diseases such as stroke patients, swallowing disorders can be predicted non-invasively rather than using existing invasive methods.

In addition, accuracy and reliability can be increased by making predictions through an artificial intelligence model using voice data and important related factor data.

Figure 1 is a schematic diagram of a swallowing disorder prediction device according to an embodiment of the present invention.
Figure 2 is a flow chart of a swallowing disorder prediction method according to an embodiment of the present invention.
Figure 3 is a table showing the relationship between physical data and diagnostic data among important associated factor data.
Figure 4 is a table showing the relationship between stroke data and diagnosis data among important associated factor data.
Figure 5 shows types of intracerebral small vessel disease.
Figure 6 is a table showing the relationship between underlying disease data and diagnosis data among important associated factor data.
Figure 7 is a table showing the relationship between lifestyle data and diagnosis data among important associated factor data.

The above objects, features and other advantages of the present invention will become more apparent by describing the preferred embodiments of the present invention in detail with reference to the accompanying drawings. In this process, the thickness of lines or sizes of components shown in the drawing may be exaggerated for clarity and convenience of explanation. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, definitions of these terms should be described based on the content throughout this specification.

Additionally, the described embodiments are provided as examples for explanation of the invention and do not limit the technical scope of the invention.

Hereinafter, with reference to the attached FIGS. 1 to 6, a device for predicting whether a swallowing disorder exists by analyzing voice patterns based on artificial intelligence according to an embodiment of the present invention will be described in detail.

As shown in Figure 1, according to the present invention, a device for predicting whether a swallowing disorder exists by analyzing voice patterns based on artificial intelligence includes an input unit 100, a conversion unit 200, a modeling unit 300, and a prediction unit. Includes part 400.

Voice data, important related factor data, and diagnostic data are input into the input unit 100.

The input unit 100 may be comprised of a first input device 110 for receiving voice data, and a second input device 120 for receiving important associated factor data and diagnostic data.

The first input device 110 may be, for example, a microphone through which voice can be input, or a device with a built-in microphone, but is not limited thereto.

The second input device 120 consists of a text or image input device for receiving important related factor data and diagnostic data, and examples include a keyboard, keypad, key button, mouse, and touch-type input device. , but is not limited to this.

Voice data, important associated factor data, and diagnostic data input through the input unit 100 will be described later.

The conversion unit 200 converts voice data among each data input to the input unit 100 into visualization data.

The converter 200 visualizes signals included in the voice data through a method or program for visualizing the voice data. This method may use, for example, Fourier synthesis or transformation, but is not limited thereto. No.

The conversion unit 200 generates visualization data by converting voice data into a wave form, spectrum, or spectrogram.

The modeling unit 300 builds a prediction model through an artificial intelligence model using learning data including the visualization data, important correlation factor data, and diagnostic data.

The modeling unit 300 uses visualization data and important related factor data among the learning data as independent variables (input variables), and the diagnostic data as dependent variables (outcome variables), and learns them through an artificial intelligence model to create a prediction model. Build it.

The artificial intelligence model used in the modeling unit 300 may be a known artificial intelligence model, but it may be preferable to use a convolutional neural network (CNN) specialized for learning visualized images.

Important associated factor data may include physical data, stroke data, underlying disease data, and lifestyle data, and will be described in detail later.

Diagnosis data is data on whether or not a swallowing disorder is diagnosed by actually performing a swallowing disorder test on a patient who has the above voice data and important associated factor data, and will be described in detail later.

This diagnostic data is a variable that is dependent on the independent variable in learning through an artificial intelligence model, but it serves to label the learning data including visualization data and important related factor data by inputting the results of the actual swallowing disorder test as described above. It is used in supervised learning to increase model accuracy. Of course, unsupervised learning is also possible.

The prediction unit 400 predicts whether there is a swallowing disorder by applying the actual prediction target patient's voice data and important associated factor data to the prediction model built in the modeling unit 300.

Next, with further reference to FIGS. 2 to 6, a method for predicting whether a swallowing disorder exists by analyzing voice patterns based on artificial intelligence according to the present invention will be described in detail.

First, learning data including voice data, important correlation factor data, and diagnosis data collected from multiple stroke patients is input (S10).

In an actual application example for performing the method according to this embodiment, 1035 pieces of voice data were collected from 423 stroke patients, 700 voice files out of the 1035 pieces of voice data were used as a learning data set, and the remaining 335 pieces of voice data were used as a learning data set. was used as a test data set for the prediction model built as described later.

The voice data, important associated factor data, and diagnostic data collected as described above are input into the input unit 100.

Specifically, voice data is input to the first input device 110, and important associated factor data and diagnostic data are input to the second input device 120.

Next, the input voice data is converted into visualization data (S20).

Voice data among the learning data input to the input unit 100 is converted into visualization data in the conversion unit 200.

The conversion unit 200 generates visualization data by converting voice data into a wave form, spectrum, or spectrogram as described above.

Next, learning data with visualization data and important related factor data as independent variables and diagnostic data as dependent variables is used by an artificial intelligence model to build a prediction model (S30).

That is, in the modeling unit 300, a prediction model is built using learning data including important related data, diagnostic data, and visualization data using an artificial intelligence model.

Here, the learning data and visualization data are data converted from voice data input to the input unit 100 by the conversion unit 200, as described above, and the important associated factor data and diagnostic data are input to the input unit 100. It's data.

In other words, the modeling unit 300 uses visualization data and important related factor data as independent variables (input variables) and diagnostic data as dependent variables (outcome variables) to perform training of an artificial intelligence model with training data to create a predictive model. Build.

Describe the important correlation factor data.

Important associated factor data include physical data, stroke data, underlying disease data, and lifestyle data.

Physical data includes age, height, weight, waist circumference, and gender data at the time of voice data collection.

As shown in Figure 3, physical data is entered using the age, height, weight, and waist circumference at the time of voice data collection as the corresponding unit values, and in the case of gender data, male can be entered as 1 and female as 2. there is.

The older you get, the higher the severity of stroke and the slower recovery after stroke.

Therefore, the older you are, the more likely it is that you will develop dysphagia due to stroke, and the more likely it is that your dysphagia will be more severe.

Additionally, because voice patterns may change depending on age, it is necessary to consider age when considering voice patterns.

Height, weight, waist circumference, etc. are indicators that reflect body mass index and obesity, and human voice can change depending on body mass index and obesity, and body mass index and obesity are closely related to the occurrence of stroke. Therefore, it is highly likely that body mass index and obesity are related to swallowing disorders.

Therefore, when considering voice patterns, it is necessary to consider body mass index and obesity level.

Additionally, because voice patterns differ greatly depending on gender, it is necessary to consider gender in order to determine whether a person has a swallowing disorder based on voice patterns.

Stroke data includes the type of stroke, size of stroke, severity of stroke, location of stroke, presence of small vessel disease, and cranial nerve examination findings.

As shown in FIG. 4, stroke data can be entered into numerical values based on predetermined standards depending on the type of stroke (cerebral infarction, cerebral hemorrhage).

Stroke can be divided into cerebral hemorrhage when a cerebral blood vessel ruptures, and cerebral infarction when a cerebral blood vessel is blocked. In general, cerebral hemorrhage is more commonly accompanied by dysphagia than cerebral infarction, so it is necessary to consider the type of stroke. As shown in Figure 4, for example, For example, cerebral infarction can be set to 1, and cerebral hemorrhage can be set to 2.

Additionally, a numerical value is input according to the size unit of the stroke, and a numerical value set according to the location where the stroke occurred can be input. Additionally, a number in a set range (eg, 1 to 5) may be entered depending on the severity of the stroke.

If the size or severity of the stroke is high, there may be an increased risk of damage to areas of the brain or cranial nerve pathways that can cause swallowing difficulties.

Therefore, in order to predict the likelihood of developing swallowing disorders, the size and severity of the stroke must be considered.

In addition to the type, size, and severity of the stroke, the most important factor in the occurrence of dysphagia is the location of the stroke.

The brainstem region is where the corticobulbar track, which is highly likely to cause swallowing disorders, is most abundant.

In other words, when a stroke occurs in the brainstem, especially in the pons or medullar region, swallowing difficulties are relatively severe and recovery is slow.

Therefore, values are entered according to the location of the stroke.

Brain small vessel disease means that pathological changes occur in small blood vessels within the brain and these pathological changes are observed on brain MRI or CT.

As shown in Figure 5, these small vessel diseases include A. white-matter hyperintensities, B. cerebral microbleeds, C. perivascular spaces, and D. asymptomatic cerebral infarction. infarctions), etc. (see white arrow). Numerical values are entered depending on the presence or type of small vessel disease.

Cranial nerve examination findings show that voice patterns may vary depending on whether there is paralysis of the 5th, 7th, 9th, 10th, 11th, or 12th cranial nerves, and are directly related to the occurrence of swallowing disorders. Therefore, values are entered according to the presence and location (left, right, bilateral) of cranial nerve paralysis.

Underlying disease data includes data on the presence or severity of high blood pressure, diabetes, hyperlipidemia, angina pectoris, myocardial infarction, metabolic syndrome, heart failure, oral disease, and vocal cord disease.

As shown in FIG. 6, the underlying disease data may be entered as 0 or 1 for each disease, or may be entered as a value in a range set according to the severity of each disease (for example, 0 to 100).

In addition, here, the presence or absence of each disease may be determined based on whether the severity value exceeds a certain value (for example, 50).

High blood pressure, diabetes, hyperlipidemia, angina pectoris, myocardial infarction, metabolic syndrome, and heart failure are important risk factors for stroke, and these underlying diseases are also closely related to small vessel disease in the brain. Small vessel disease is a type of brain disease. Microscopic damage reflects damage to the brain's neural network.

The presence of an underlying disease itself is closely related to the occurrence and severity of stroke. Nerve network damage, stroke occurrence, and severity of stroke are closely related to dysphagia due to stroke, so in order to predict the presence and severity of dysphagia, it is necessary to consider the presence of underlying diseases and oral diseases (oral inflammation, cancer, Other diseases, etc.) and vocal cord diseases (vocal cord paralysis, vocal cord nodules, cancer, other diseases, etc.) are closely related to swallowing disorders, so values are entered according to their presence and severity.

Lifestyle data includes the presence or frequency of smoking and alcohol consumption, and the presence, frequency or type of drug consumption.

As shown in Figure 7, the lifestyle data includes smoking status or frequency (packs/day), alcohol consumption or frequency (times/week), and drug intake and frequency (times/week). A set value can be entered depending on the type of drug being administered.

Smoking and alcohol consumption are important risk factors for stroke, and as the amount of smoking and alcohol consumption increases, the risk of stroke increases and the severity of stroke also increases.

The higher the severity of the stroke, the more likely it is to be accompanied by dysphagia. Therefore, when diagnosing or measuring the severity of dysphagia accompanying stroke, values are entered according to the frequency of smoking and alcohol consumption.

In stroke patients, antidepressants, tranquilizers, and dementia behavioral disorder control drugs can lower the level of consciousness, trigger extrapyramidal symptoms, or cause deterioration of cranial nerve function, which can cause or worsen swallowing disorders. To check the occurrence and severity of a disorder, values are entered for the type and frequency of medication being taken.

Diagnosis data is data on the presence or severity of swallowing disorder diagnosed through actual examination of the above-mentioned stroke patients.

Diagnosis of swallowing disorders is performed by asking stroke patients to swallow water, which is a test method currently used for swallowing disorders, or by performing a video fluoroscopic swallowing test to check whether or not swallowing is possible. Also, if a sound is heard while swallowing water 10 times. The diagnosis was made by checking the number of cases.

As shown in Figures 3 to 7, the diagnostic data may be entered as 0 or 1 for whether the swallowing disorder is present, or a value in a range set according to the severity of the swallowing disorder (e.g., 0 to 100) may be entered, Here, the presence or absence of a swallowing disorder can be determined based on whether the severity level exceeds a certain number (for example, 50).

The actually diagnosed diagnostic data is directly input as the dependent variable of the learning data. This is a supervised learning method among artificial intelligence learning methods, and the resulting dependent variable is input along with the independent variable.

In the case of unsupervised learning, the artificial intelligence model must find the relationship function with the dependent variable using only the input independent variables, so accuracy is somewhat low, learning time is slow, and a large amount of learning data is required.

In the case of supervised learning, learning is performed by inputting dependent variables according to the independent variables together, so accuracy is high and learning time is fast, and it is possible to build a high-accuracy model with only a relatively small amount of learning data.

When artificial intelligence modeling was not applied to the learning data, when cases with swallowing disorders were applied as the dependent variable, the AUC was 0.83, and when artificial intelligence modeling according to this example was used, the AUC was 0.87, significantly increasing the predictive power. was confirmed.

Afterwards, a test was performed using 335 voice files, excluding the 700 data used to build the prediction model, among the 1035 data mentioned above.

In tests using the test set, it was confirmed that the prediction model built using the learning data was valid as shown below.

Training dataset: AUC 0.89, sensitivity 0.83, specificity 0.93

Test dataset: AUC 0.87, sensitivity 0.81, specificity 0.92

Next, the presence or absence of swallowing disorder is predicted by applying prediction target data including voice data and important associated factor data of the patient to be predicted to the constructed prediction model (S40).

The prediction unit 400 inputs the patient's voice data and important associated factor data, that is, the independent variable that is actually the prediction target, into the prediction model built in the modeling unit 300 as described above to predict whether the dependent variable is dysphagia. And through this, it is possible to predict whether the actual patient to be predicted has a swallowing disorder.

Here too, as a prediction result, the actual presence or absence of swallowing disorder may be predicted as 0 or 1, or may be predicted as a value in a range set according to the severity of the swallowing disorder (e.g., 0 to 100), where the presence or absence of swallowing disorder may be determined according to the severity. It can be determined depending on whether the value exceeds a certain value (for example, 50).

The device for predicting whether a swallowing disorder exists by analyzing voice patterns based on artificial intelligence according to the present invention may be implemented by a computing device that includes at least some of a processor, memory, user input device, and presentation device.

These computing devices may include various devices such as smartphones, tablets, laptops, desktops, servers, and clients.

Memory is a medium that stores computer-readable software, applications, data, etc. that are coded to perform specific tasks when executed by a processor.

The processor can read and execute computer-readable software, applications, data, etc. stored in the memory.

In addition, the method of predicting whether a swallowing disorder exists by analyzing speech patterns based on artificial intelligence according to the present invention can be executed by the above-mentioned computing device, and to realize this, a computer is stored in a storage medium that can be read by the computing device. It can be provided as a program and a storage medium that stores it and can be read by a computer.

According to the device and method for predicting swallowing disorders by analyzing voice patterns based on artificial intelligence according to the present invention, serious complications such as aspiration pneumonia, which is the main cause of death in patients with neurological diseases such as stroke patients, can be prevented using existing invasive methods. This can be predicted non-invasively.

In addition, accuracy and reliability can be increased by making predictions through an artificial intelligence model using voice data and important related factor data.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to the specific embodiments described above. In other words, a person skilled in the art to which the present invention pertains can make numerous changes and modifications to the present invention without departing from the spirit and scope of the appended claims, and all such appropriate changes and modifications can be made. Equivalents should also be considered to fall within the scope of the present invention.

100: input unit
200: conversion unit
300: Modeling department
400: prediction unit

Claims (6)

A device that predicts swallowing disorders based on artificial intelligence,
An input unit 100 into which voice data, important associated factor data, and diagnostic data of a plurality of stroke patients are input;
A conversion unit 200 that converts voice data input to the input unit 100 into visualization data;
A modeling unit 300 that constructs a prediction model through an artificial intelligence model using learning data that uses the visualization data and the important correlation factor data as independent variables and the diagnostic data as a dependent variable; and
It includes a prediction unit 400 that predicts whether there is a swallowing disorder by applying prediction target data including voice data and important associated factor data of the prediction target patient to the prediction model,
The important correlation factor data is,
Physical data including age, height, weight, waist circumference, and gender data at the time of measuring the voice data,
Stroke data including type of stroke, size of stroke, severity of stroke, data on location of stroke, presence and type of small vessel disease, and data on cranial nerve examination findings;
Underlying disease data, including data on the presence or severity of high blood pressure, diabetes, hyperlipidemia, angina pectoris, myocardial infarction, metabolic syndrome, heart failure, oral disease, and vocal cord disease;
Includes lifestyle data, including data on the presence or frequency of smoking and alcohol consumption, and data on the presence, frequency, or type of drug consumption;
The diagnostic data includes data on the presence or severity of a swallowing disorder diagnosed through an actual swallowing disorder examination,
The important associated factor data and the diagnostic data are quantified and input according to preset standards,
Device.
According to claim 1,
The artificial intelligence model is a convolutional neural network (CNN),
Device.
As a method of predicting whether there is a swallowing disorder using the device according to claim 1,
(a) inputting voice data, important associated factor data, and diagnostic data collected from multiple stroke patients into the input unit 100;
(b) converting the input voice data into visualization data in the conversion unit 200;
(c) By the modeling unit 300, learning data with the visualization data and the important correlation factor data as independent variables and the diagnostic data as dependent variables is used by an artificial intelligence model to build a prediction model. step; and
(d) applying prediction target data including voice data and important associated factor data of the patient to be predicted to the constructed prediction model to predict whether or not there is a swallowing disorder by the prediction unit 400;
Important correlation factor data included in the learning data and the prediction target data, respectively,
Physical data including age, height, weight, waist circumference, and gender data at the time of collecting the voice data,
Stroke data including type of stroke, size of stroke, severity of stroke, data on location of stroke, presence and type of small vessel disease, and data on cranial nerve examination findings;
Underlying disease data, including data on the presence or severity of high blood pressure, diabetes, hyperlipidemia, angina pectoris, myocardial infarction, metabolic syndrome, heart failure, oral disease, and vocal cord disease, and
Includes lifestyle data, including data on the presence or frequency of smoking and alcohol consumption, and data on the presence, frequency, or type of drug consumption;
The diagnostic data includes data on the presence or severity of a swallowing disorder diagnosed through an actual swallowing disorder examination,
The important associated factor data and the diagnostic data are quantified and input according to preset standards,
method.
According to claim 3,
The artificial intelligence model is a convolutional neural network (CNN),
method.
A storage medium storing a computer program for executing the method according to claim 3 or 4.
A computer program stored in a storage medium for executing the method according to claim 3 or 4.

Images