KR20220021975A - System for providing non-face-to-face neurological disease screening service - Google Patents

Abstract

The present invention relates to a system for providing a non-face-to-face neurological disease diagnosis service to increase classification accuracy and efficiency by type of neurological diseases. According to the present invention, the system comprises: a user terminal using a camera to take a diagnosis image including a user's face and providing diagnosis request information including the captured diagnosis image; and a diagnosis service providing server using a neurological disease diagnosis model to detect the user's face in the diagnosed image and extract an eyeball region when the diagnosis request information is received, extracting a pupil position from the extracted eyeball region to track gaze direction information, using the tracked gaze direction information and eye movement analysis information to diagnose a neurological disease state, and providing a diagnosis result of the neurological disease state to the user terminal. The neurological disease diagnosis model uses a clinical database collecting and storing clinical data on neurological diseases including dementia and Parkinson's disease to form an artificial intelligence (AI)-based neural network to classify the neurological diseases through image preprocessing and learning and inputs the diagnosed image to the AI-based neural network to output diagnosis results including types of neurological diseases and matching probabilities for each type.

Description

System for providing non-face-to-face neurological disease screening service

The present invention relates to a non-face-to-face neurological disease screening service providing system that provides diagnostic results including types and probabilities of neurological diseases based on artificial intelligence.

The content described in this section merely provides background information on an embodiment of the present invention and does not constitute the prior art.

Due to the aging of the population, the number of patients with senile dementia in Korea is expected to exceed 750,000 as of 2018, and the prevalence of dementia is reported to reach 10% of those aged 65 and over. In addition, according to the 2017 Annual Report of the Central Dementia Center, 1 in 4 people aged 80 and over and 1 in 2.5 people over 85 years old, dementia has already become a very common disease in Korea. The annual management cost per dementia patient was estimated at 20.54 million won, and the national dementia management cost was 13.6 trillion won, accounting for about 0.8% of GDP. In order to reduce these social costs, a system for early diagnosis of dementia is an essential technology.

However, in clinical clinical dementia tests, the diagnosis is confirmed when dementia has already progressed and brain atrophy is observed, so early treatment is often missed.

Researchers in the UK are conducting a study to predict patients with early dementia by analyzing eye movement, which indicates that tau protein is deposited in the locus ceruleus of the brainstem in the early stages of dementia and is related to saccadic movement. This is a study focusing on the mechanism of lowering synaptic function. The above study requires a specific sensor for tracking eye movement and additional equipment, and has a limitation in that it is difficult to continuously follow and observe during daily life.

In order to solve the above problems, the present invention collects clinical data for pupil movement analysis for diagnosing neurological diseases such as dementia and Parkinson's disease according to an embodiment of the present invention and uses machine learning to diagnose neurological diseases. The purpose is to improve the classification accuracy and efficiency by type.

However, the technical task to be achieved by the present embodiment is not limited to the technical task as described above, and other technical tasks may exist.

As a technical means for achieving the above technical problem, the system for providing a non-face-to-face neurological disease diagnosis service according to an embodiment of the present invention uses a camera to capture an image to be diagnosed including a user's face, and the photographed to be diagnosed a user terminal providing diagnosis request information including an image; and when the diagnosis request information is received, the user's face is detected from the image to be diagnosed using the neurological disease diagnosis model to extract the eye position region, and the pupil position is extracted from the extracted eye position region to obtain gaze direction information. and a diagnostic service providing server that tracks, diagnoses a neurological disease state using the tracked gaze direction information and eye movement analysis information, and provides a diagnosis result for the neurological disease state to the user terminal, wherein the The neurological disease diagnosis model collects and stores clinical data on neurological diseases including dementia and Parkinson's disease, and uses an artificial intelligence-based neural network to classify neurological diseases through image preprocessing and learning. Network) and inputting the image to be diagnosed to the artificial intelligence-based neural network, and outputting a diagnosis result including a type of neurological disease and a matching probability for each type.

The diagnosis request information includes at least one basic information of the user's age, gender, symptoms, and questionnaire data.

The diagnosis service providing server provides an application for screening a neurological disease to the user terminal, and the user terminal displays a target on the screen through the application for screening a neurological disease, and the user moves the eye according to the movement of the target. The gaze task and the opposite gaze gaze operation of moving the eyeball in the opposite direction to the target while the target is stationary are performed at least once or more, and the neurological disease diagnosis model performs the gaze gaze operation opposite to the gaze gaze operation The eye movement is coordinated through the coordinated eye movement, and the movement time and movement speed of the eyeball are extracted through the coordinated eye movement to calculate the eye movement analysis information.

The neurological disease diagnosis model calculates eye movement analysis information including eye variability and pupil position information using the movement time and movement speed of the eyeball, and when the eye variability changes by more than a preset threshold, it deviates from the normal range It is determined that there is a medical condition.

The neurological disease diagnosis model recognizes the user's face in the image to be diagnosed through a machine learning-based algorithm, extracts landmarks constituting the eyeball to detect the eyeball position from the recognized user's face, and performs image processing Through this method, the pupil position and the central position of the eye are extracted from the landmark of the eye to track the eye movement.

According to the above-described problem solving means of the present invention, the present invention provides an artificial intelligence-based neurological disease diagnosis model when a user captures and transmits an image to be diagnosed including a user's face using a camera mounted on a user terminal such as a smartphone. can provide users with early diagnosis of neurological disease types from the clinical database, and when a specific neurological disease is identified, the diagnosis result and user information are shared with an expert terminal with the user's consent to efficiently provide customized medical services for each user can provide

1 is a view for explaining a non-face-to-face neurological disease examination service providing system according to an embodiment of the present invention.
2 is a view for explaining a disease detection process of a neurological disease diagnosis model according to an embodiment of the present invention.
3 is an exemplary diagram of extracting a pupil through a neurological disease diagnosis model according to an embodiment of the present invention.
4 is a view for explaining a process of analyzing an eye movement in an image to be diagnosed in a non-face-to-face neurological disease diagnosis system according to an embodiment of the present invention.
FIG. 5 is a graph showing the results of the eye movement analysis of FIG. 4 .
6 is a graph illustrating eye movements of a dementia patient.
7 is a graph illustrating eye movement of a Parkinson's disease patient.
8 is an exemplary view for explaining a diagnosis result provided by a neurological disease diagnosis model according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

In the present specification, a 'terminal' may be a wireless communication device with guaranteed portability and mobility, for example, any type of handheld-based wireless communication device such as a smart phone, tablet PC, or notebook computer. In addition, the 'terminal' may be a wired communication device such as a PC that can connect to another terminal or server through a network. In addition, the network refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW). : World Wide Web), wired and wireless data networks, telephone networks, and wired and wireless television networks.

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

The following examples are detailed descriptions to help the understanding of the present invention, and do not limit the scope of the present invention. Accordingly, an invention of the same scope performing the same function as the present invention will also fall within the scope of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view for explaining a non-face-to-face neurological disease examination service providing system according to an embodiment of the present invention.

Referring to FIG. 1 , the neurological disease examination service providing system includes, but is not limited to, a user terminal 100 , a diagnosis service providing server 200 , and an expert terminal 300 .

The user terminal 100 captures an image to be diagnosed including the user's face using a camera, and transmits the photographed image to be diagnosed and diagnosis request information to the diagnosis service providing server 200 . Here, the user terminal 100 may be a terminal possessed by the user or a terminal possessed by a guardian according to the degree of the user's cognitive ability.

In this case, the diagnosis request information may include any one or more basic information among the user's age, gender, symptoms, and questionnaire data. The questionnaire data may be data that can confirm cognitive ability or symptoms in a simple question-and-answer format using voice recognition technology or fingertip (or finger) detection technology. Such questionnaire data can be collected in various forms, such as a question-and-answer format using text input or pupil movement extraction.

When the diagnosis request information is received from the user terminal 100, the diagnosis service providing server 200 detects the user's face from the image to be diagnosed using the neurological disease diagnosis model, extracts the eyeball location area, and extracts the extracted eye location area. extracts the pupil position from , and tracks the gaze direction information, diagnoses a neurological disease state using the tracked gaze direction information and eye movement analysis information, and provides the diagnosis result for the neurological disease state to the user terminal 100 do. The diagnosis service providing server 200 may provide an application for non-face-to-face neurological disease diagnosis to the user terminal 100 , and may exchange real-time information with the user terminal 100 through the application.

The diagnostic service providing server 200 may be a computer body for a server in a general sense, or may be implemented as various types of devices capable of performing a server role. Specifically, the diagnosis service providing server 200 may be implemented in a computing device including a communication module (not shown), a memory (not shown), a processor (not shown), and a database (not shown).

The expert terminal 300 is managed by a person with specialized knowledge related to a neurological disease, and can provide customized medical services for each patient by synthesizing the diagnosis results of the neurological disease diagnosis model and treatment request information, and provides diagnostic services The server 200 may provide expert knowledge with the user terminal 100 in various communication methods such as video chatting, text chatting, and mail. The treatment request information may include any one or more basic information among the user's age, gender, symptoms, and questionnaire data.

That is, the user terminal 100 transmits treatment request information for professional diagnosis or treatment to the diagnosis service providing server 200 based on the diagnosis result, and the diagnosis service providing server 200 sends the diagnosis service providing server 200 to the expert terminal ( 300), the diagnosis result and treatment request information may be transmitted, and a one-to-one chatting window may be provided for treatment linkage between the user terminal 100 and the expert terminal 300 at a preset time. In this case, the treatment request information may include the user's age, gender, symptoms, treatment type (eg, visit treatment, online treatment, etc.), whether information is disclosed or not. On the other hand, the diagnosis service providing server 200 stores the basic information of the user terminal 100 and the specialized field information of the expert terminal 300 in a database, and the expert terminal 300 suitable for the type of neurological disease according to the diagnosis result. is assigned to the user terminal 100 , and only the assigned expert terminal 300 provides the diagnosis result of a neurological disease including basic information of the user terminal 100 .

In the above, the user terminal 100 and the expert terminal 300 may be implemented as a mobile phone, a TV, a PDA, a tablet PC, a PC, a notebook PC, and other user terminal devices.

2 is a diagram for explaining a disease detection process of a neurological disease diagnosis model according to an embodiment of the present invention, and FIG. 3 is an exemplary view of extracting a pupil through a neurological disease diagnosis model according to an embodiment of the present invention. am.

2 and 3 , the neurological disease diagnosis model collects and stores clinical data related to eye movement for neurological diseases including dementia and Parkinson's disease through image preprocessing and learning using the clinical database 250 . An artificial intelligence-based neural network is configured to classify neurological diseases, and a diagnostic image is input to the artificial intelligence-based neural network to output diagnosis results including types of neurological diseases and matching probabilities for each type.

Artificial intelligence-based neural networks include Support Vector Machine (SVM), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long-Short Term Memory, A classification model based on feature extraction such as LSTM) can be used.

The neurological disease diagnosis model is largely a face recognition process that recognizes a user's face from an image to be diagnosed through a machine learning-based algorithm, and landmark extraction that extracts landmarks constituting the eyeball to detect the position of the eyeball from the recognized user's face It consists of an eye movement tracking process that extracts the pupil and the center position of the eye from the landmark of the eye through image processing.

First, in the face recognition process, using HoG (Histogram of Oriented Gradients) features and SVM-based face recognition algorithm, efficient face recognition is performed with high speed, low resource requirements, and high accuracy. The face detection method extracts the features of the object of interest (that is, the user's face part) from the input diagnostic image, generates SVM learning and a classifier using the extracted feature data, and uses the generated classifier to extract the features of interest from the input image. As an object of interest is searched by extracting an object similar to the characteristic of the object, the time and effort required to manually search the object of interest in a wide area can be reduced because the object of interest is automatically found.

Next, in the landmark extraction process, a facial landmark extraction algorithm is used to recognize the position of the eyes in the face recognized in the face recognition process. At this time, the face landmark extraction algorithm is a regression tree-based machine learning algorithm that extracts 68 landmarks from the recognized face with millisecond speed and high accuracy. The landmark becomes the landmark corresponding to the left and right eyes, respectively. In the eye movement tracking process, the eye area is specified from the extracted landmark through the Convex Hull algorithm.

As shown in FIG. 3 , the process of extracting the pupil from the extracted eye region is performed through masking based on a threshold value. That is, the neurological disease diagnosis model can set a threshold value in order to respond to environmental changes according to the lighting, shooting angle, and shooting location of an inputted image to be diagnosed, and based on the set threshold value, the pupil and non-pupil in the eye region parts can be separated.

Therefore, the neurological disease diagnosis model applies the adaptive threshold technique, enabling optimal pupil extraction and eye movement analysis even under changes in external environment and lighting. When the pupil part is extracted from the eye, the gaze direction information can be tracked through the relative position of the central point position (X) of the eye and the central point position (+) of the pupil, and the tracked pupil position is stored in a database in real time, This can be used as clinical data in the future.

4 is a view for explaining a process of analyzing an eye movement in an image to be diagnosed in a non-face-to-face neurological disease diagnosis system according to an embodiment of the present invention, and FIG. 5 is a graph illustrating the result of the eye movement analysis of FIG. 4 . will be. 6 is a graph illustrating eye movement of a patient with dementia, FIG. 7 is a graph illustrating eye movement of a patient with Parkinson's disease, and FIG. 8 is a diagnosis result provided by a neurological disease diagnosis model according to an embodiment of the present invention. It is an example diagram to explain.

4 and 5 , the diagnosis service providing server 200 provides a point at a preset position in the left and right direction of the screen while the user gazes at the camera of the user terminal 100 a certain number of times or more through the neurological disease diagnosis application. Train your eyes to follow them.

As an example, the diagnostic service providing server 200 performs a gaze gaze operation to gaze at the opposite side with a screen (or voice) while the dot is stopped at the left end, and at this time, the opposite gaze gaze from the left end to the right end By performing the task, the movement time to the pupil movement in the opposite direction and the eye movement speed are measured. Contrary to the above operation, the diagnostic service providing server 200 performs a gaze gaze operation to gaze at the opposite side while the point is stopped at the right end, and a gaze gaze operation in the opposite direction to gaze from the right end to the left end to increase movement time and Measure the eye movement speed. The measured eye movements can be coordinated and expressed as shown in Table 1, and eye movement analysis information including eye variability and pupil position information can be obtained using the eye movement time and eye movement speed extracted through coordinated eye movement. can be extracted.

target Left eye (x, y) right eye (x, y) right end +1 0 +1 0 middle 0 0 0 0 left end -One 0 -One 0 above the center 0 +1 0 +1 below the center 0 -One 0 -One

As shown in FIG. 6 , when looking at the eye movements of the dementia patient through the gaze gaze task according to the movement of a target such as a point or a picture on the screen of the user terminal 100 and the gaze gaze task in the opposite direction, a decrease in the prefrontal inhibition function As a result, it can be seen that a predetermined delay time occurs when the pupil follows the target or gazes in the opposite direction without the user's knowledge.

Here, saccade is a conjugate eye movement that plays the most important role in daily activities, such as exploring the surrounding environment or reading a book, responsible for the function of quickly moving the gaze from the currently focused object to another object. The function of the saccade is evaluated by the range of eye movement, speed, latency, and accuracy. The maximum horizontal range of eye movement in young adults is about 50 degrees. The vertical eye movement range gradually decreases with age, reaching 45 degrees (upper, 43 degrees; downward, 47 degrees) when young and about 33 degrees in older adults. Incubation period The time it takes from the appearance of the target to the occurrence of saccade is around 200ms in normal people. The incubation period increases with age because the processing time in the central nervous system is delayed. If you remove the object you are currently looking at before the new target appears, the latency of the saccade to the new target is shortened.

Therefore, the neurological disease diagnosis model is that in the context of brain tumor, stroke, and other neurological diseases, an increase in neoplasia or hemorrhage and cerebral edema in the limited two spaces results in an increase in intracranial pressure. , 4, and 6 dysfunction, restricting eye movement, and scoring the inability to follow the point movement in both sides, up and down, and both oblique lines can be used to screen for neurological diseases such as brain tumors and strokes.

In the neurological disease diagnosis model, in Parkinson's disease, muscle tremor occurs due to a lack of dopamine in the brain. Parkinson's disease is screened when the variability is above a predetermined value.

As shown in FIG. 7 , in the case of a Parkinson's disease patient, when eye movements are recorded through eye movement analysis information, the eye movements are not continuous in a linear-circular manner, but are cut off in a step form and variability occurs. Therefore, in the case of Parkinson's disease, the neurological disease diagnosis model can screen for diseases because eye tremors are sustained in the process of performing the gaze-gaze task, and eye movement variability changes more than predetermined in the process of gaze-gaze in the opposite direction. .

Neurological disease diagnosis model shows that in the case of mild cognitive impairment and dementia, amyloid beta and tau proteins are deposited in specific regions of the brain. Responsible for the function of gaze and visual field gaze, and based on eye movement, pupil movement, and contralateral gaze task, the success or failure of the contralateral gaze task based on a predetermined time or longer, and the stress index through changes in pupil size and facial blood flow are scored for mild recognition. Screened for disability and dementia.

When the gaze is moving along a specific object, the locus nuclei of the brainstem, the center of visual observation, fix the eye movement to the movement of a specific object. Inhibitory signal transduction of the frontal lobe is required for the locus nuclei of the brain .

As such, the neurological disease diagnosis model uses artificial intelligence technology to infer the disease type, probability, etc. from the image to be diagnosed, then visualizes it and provides the diagnosis result to the user terminal 100 . That is, as shown in FIG. 8, the neurological disease diagnosis model displays the eye movement range in the user's face image that has been subjected to personal information protection processing with the user terminal 100, and the probability of dementia is 82%, the probability of Parkinson's disease is 11%, Others indicate a 7% chance of concussion and provide a pre-diagnosis result that allows you to choose to consult your doctor one-on-one or rediagnose.

As described above, the present invention accurately extracts the position of the eyeball and the eye movement from the image to be diagnosed taken through the camera of the user terminal 100 such as a smartphone, and responds to the movement of the target displayed on the screen of the user terminal 100 By recording the movement of the eye, it is possible to distinguish normal people from dementia patients.

On the other hand, in the present invention, the LPT wave and heart rate are extracted from the user's face and fingers through a camera mounted on the front or rear side of the user terminal 100 , and the rate of change of sympathetic/parasympathetic nerves is measured through the extracted LPT wave and heart rate. It is also possible to differentiate between normal and dementia patients by calculation.

In addition, the diagnosis service providing server 200 builds clinical data such as gaze direction information and eye movement analysis information with the user's consent, and classifies differences between groups into SVM as dimensionally reduced data using a self-encoder, etc. Time series from a classification model Early dementia diagnosis can be performed by extracting data and applying it to RNN and LSTM.

The embodiments of the present invention described above may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Such recording media includes computer-readable media, and computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Computer readable media also includes computer storage media, which include volatile and nonvolatile embodied in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. , including both removable and non-removable media.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: user terminal
200: diagnostic service providing server
300: expert terminal

Claims (5)

a user terminal for photographing an image to be diagnosed including the user's face using a camera and providing diagnosis request information including the photographed image to be diagnosed; and
When the diagnosis request information is received, the user's face is detected from the image to be diagnosed using the neurological disease diagnosis model to extract the eye position region, and the pupil position is extracted from the extracted eye position region to track the gaze direction information. and a diagnosis service providing server that diagnoses a neurological disease state using the tracked gaze direction information and eye movement analysis information, and provides a diagnosis result for the neurological disease state to the user terminal,
The neurological disease diagnosis model,
Constructing an artificial intelligence-based neural network to classify neurological diseases through image preprocessing and learning using a clinical database that collects and stores clinical data on neurological diseases, including dementia and Parkinson’s, and A non-face-to-face neurological disease diagnosis service providing system that inputs the diagnosed image to an artificial intelligence-based neural network and outputs a diagnosis result including a type of neurological disease and a matching probability for each type. According to claim 1,
The diagnosis request information is
A non-face-to-face neurological disease screening service providing system that includes any one or more basic information of the user's age, gender, symptoms, and questionnaire data. According to claim 1,
The diagnosis service providing server provides an application for neurological disease examination to the user terminal,
The user terminal makes the target appear on the screen through the application for neurological disease screening, and the user moves the eyeball according to the movement of the target in the opposite direction of the gaze-gaze operation and the target in the opposite direction of the target when the target is stopped. to perform the gaze-gaze task at least once,
The neurological disease diagnosis model coordinates eye movements through the gaze-gaze task and the opposite-direction gaze-gaze task, and extracts the movement time and movement speed of the eyeball through the coordinated eye movement to calculate eye movement analysis information. A non-face-to-face neurological disease screening service providing system. 4. The method of claim 3,
The neurological disease diagnosis model calculates eye movement analysis information including eye variability and pupil position information using the movement time and movement speed of the eyeball, and when the eye variability changes by more than a preset threshold, it deviates from the normal range. A non-face-to-face neurological disease screening service providing system that determines that there is a medical disease. According to claim 1,
The neurological disease diagnosis model,
Recognizes the user's face in the image to be diagnosed through a machine learning-based algorithm, extracts landmarks constituting the eyeball to detect the eyeball position from the recognized user's face, and uses image processing to position the pupil from the landmark of the eye A non-face-to-face neurological disease screening service providing system that tracks eye movements by extracting the central position of the and eye.

Images