KR102585321B1 - Method of learning medical image for dementia decision - Google Patents

Abstract

A medical image learning method for determining dementia is disclosed. The present invention includes the steps of generating first data based on medical images and labeling data for determining dementia, learning the medical images using a first learning algorithm and the first data, and using a test image to learn the medical images. It includes a step of verifying the results of learning, and the medical image may be an eye image captured with a fundus camera.

Description

Medical image learning method for dementia judgment {METHOD OF LEARNING MEDICAL IMAGE FOR DEMENTIA DECISION}

The present invention relates to a medical image learning method for determining dementia. More specifically, it relates to a method for preprocessing and learning eye images in order to learn eye images using a CNN learning algorithm.

Among degenerative brain diseases, dementia in particular is a neurological disease so important that it is currently called one of the four major causes of death. Symptoms appear only after the condition has worsened, and the symptoms cannot be accurately identified, making early detection difficult.

Dementia is a representative brain disease caused by damage to brain cells, and it mainly occurs in old age. Unlike other cells, brain cells do not regenerate, so prevention or early detection and treatment are the safest and most efficient ways to treat it. It is known.

Dementia refers to a condition that causes difficulties in daily life or social life due to disorders in the temporal lobe and frontal lobe executive function, including memory, attention, language, and visuospatial ability, which are various cognitive functions that humans have. Depending on the cause, dementia is classified into Alzheimer's dementia, vascular dementia, Lewy body dementia, and frontal lobe dementia. Alzheimer's dementia accounts for 70% of dementia patients.

Alzheimer's dementia causes a gradual decline in cognitive ability and often begins with memory loss. The cause is that abnormal protein clumps called amyloid plaques and protein clumps called neurofibrillary tangles become intertwined, blocking communication with nerve cells and killing the cells, resulting in Alzheimer's dementia.

Vascular dementia is a cognitive impairment caused by damage to blood vessels in the brain. Therefore, there is a possibility that stroke patients may develop dementia. In addition, symptoms similar to Alzheimer's disease may appear, so it may not be easy to distinguish it.

Lewy body dementia can occur when abnormal protein clumps called alpha-cytuclein develop in specific areas of the brain. Lewy body dementia can cause changes in movement, thinking, and behavior. Typically, if movement symptoms appear first, it can be diagnosed as Parkinson's disease, and if cognitive symptoms appear first, it can be diagnosed as Lewy body dementia.

Frontal lobe dementia, also called frontotemporal dementia, can occur when there is progressive damage to the frontal and temporal lobes of the brain. Damage to the frontal lobe can result in behavioral symptoms and personality changes, and damage to the temporal lobe can cause language impairment, and these two can occur simultaneously.

The iris is an extension of the brain and has hundreds of thousands of nerve endings, capillaries, and muscle tissue, and it is known that the iris can serve as a diagnostic index for the brain. Using this, diagnostic methods using iris are being developed to examine personal health or response to treatment.

In particular, in the past, even though dementia was predicted based on deep learning, research was being conducted to determine dementia by learning microorganisms and viruses distributed in the patient's intestines, rather than images of the iris. However, this method of determining dementia was conducted. There were problems in that it was inefficient and difficult to implement.

Republic of Korea Patent Publication No. 10-2020-0066578 (published on June 10, 2020)

The purpose of the present invention is to provide a medical image learning method for dementia determination in order to more easily and quickly determine dementia by learning medical images (particularly eye images) using artificial intelligence.

In addition, the purpose of the present invention is to provide a medical image learning method for dementia judgment to significantly reduce pre-processing time by labeling image files based on their names for rapid pre-processing of vast medical images.

In addition, the purpose of the present invention is to provide a medical image learning method for dementia judgment that labels the name of the image file using labeling data of a CSV file for rapid preprocessing of a vast amount of medical images.

Additionally, the purpose of the present invention is to provide a medical image learning method for dementia judgment that can produce more accurate results by providing a learning algorithm that takes into account the patient's age.

In order to solve the above-described problem, the present invention includes generating first data based on medical images and labeling data for determining dementia, learning the medical image using a first learning algorithm and the first data. and verifying the results of the learning using a test image, wherein the medical image may be an eye image captured with a fundus camera.

Additionally, in the step of generating the first data, the first data may be generated by matching the file name of the medical image and the labeling data.

At this time, the labeling data includes data for binary classification, and the step of generating the first data may generate the first data by matching the file name of the medical semantic paper and the data for binary classification.

Additionally, the step of learning the medical image may generate an embedding value for the medical image and learn features of the medical image based on the embedding value and the first data.

At this time, the first learning algorithm may be a learning algorithm using a plurality of layers according to a CNN learning algorithm.

At this time, the first learning algorithm includes a Siamese Network algorithm and a Logistic regression analysis algorithm, wherein the Siamese network includes a first network and a second network, and the first network includes the It is replaced with an embedding value, the second network obtains a new embedding value for the new medical image, and the logistic regression analysis algorithm can perform binary classification based on the embedding value and the new embedding value.

Additionally, the embedding value may be modified based on the new embedding value.

In addition, in order to solve the above-described problem, the present invention provides a non-transitory computer-readable medium storing instructions, which, when executed by a processor, cause the processor to perform any one of the above-described methods. It may be a non-transitory computer-readable medium that allows.

The present invention has the effect of providing a medical image learning method for dementia determination in order to more easily and quickly determine dementia by learning medical images (especially eye images) using artificial intelligence.

Additionally, the present invention has the effect of enabling faster preprocessing of vast medical images by performing binary classification labeling on the names of image files.

In addition, the present invention has the effect of enabling faster preprocessing of vast medical images by applying labeling data included in the CSV file to the name of the image file.

In addition, the present invention proposes a learning algorithm that takes into account the patient's age, thereby correcting errors due to eye aging, enabling more efficient and faster preprocessing of vast medical images.

The effects that can be obtained according to the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a diagram showing a medical image learning method for determining dementia according to the present invention.
Figure 2 is a diagram showing a medical image and its file name according to the present invention.
Figure 3 is a diagram showing a portion of a CSV file according to the present invention.
Figure 4 is a diagram showing a medical image classification and algorithm verification method according to the present invention.
Figure 5 is a diagram showing the CNN network algorithm according to the present invention.
Figure 6 is a diagram showing a CNN filter according to the present invention.
Figure 7 is a diagram showing the Siamese network algorithm according to the present invention.
The accompanying drawings, which are included as part of the detailed description to aid understanding of the present specification, provide embodiments of the present specification and explain technical features of the present specification together with the detailed description.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted.

Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted.

In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Below, based on the above-described contents, a detailed description of the medical image learning method for determining dementia according to a preferred embodiment of the present specification is as follows.

1 is a diagram showing a medical image learning method for determining dementia according to the present invention.

According to Figure 1, the learning method according to the present invention includes generating first data based on medical images and labeling data for determining dementia (S1100), and generating a medical image using the first learning algorithm and first data. It may include a step of learning (S1200) and a step of verifying the learning result using a test image (S1300).

At this time, the medical image may be an eye image captured with a fundus camera. The eye image may include at least one of the iris, blood vessels distributed to the eye, and nerves distributed to the eye. The learning method according to the present invention can learn at least one of the iris, blood vessels distributed in the eye, and nerves distributed in the eye among the above-described eye images.

Additionally, the step of generating first data based on the medical image and labeling data for determining dementia (S1100) may be a step of generating the first data by matching the file name and labeling data of the medical image.

At this time, the labeling data includes data for binary classification, and the step of generating the first data may generate the first data by matching the file name of the medical semantic paper and the data for binary classification.

In the step of learning the medical image (S1200), an embedding value for the medical image may be generated and features of the medical image may be learned based on the embedding value and the first data. That is, the step of learning the medical image (S1200) may be a step of learning the embedding value for the medical image based on the labeled result according to the first data.

The step of verifying the learning results using a test image (S1300) may be a step of determining whether the learning was successful. The step of verifying the results of learning (S1300) is to perform binary classification by inputting a test image for which the result of dementia is known in advance, and to compare the binary classification result for the test image with the previously known binary classification result. It may include steps. If the binary classification result for the test image is the same as the previously known binary classification value, it is determined that the learning is being performed well, and verification can be terminated. Conversely, if the two binary stream values are different from each other, it is determined that the conditions for the corresponding learning were set incorrectly, and verification may be terminated.

Additionally, the step of verifying the learning results (S1300) may be performance evaluation. That is, the step of verifying the results of learning (S1300) involves inputting a plurality of test images and obtaining binary classification results for the plurality of test images, and combining the obtained binary classification results and previously known binary classification values. It may include a step of comparing, and a step of obtaining a probability value for a comparison result of a plurality of test images. Therefore, if the obtained probability value is greater than a predetermined value, the result of the learning may be determined to be verified.

Figure 2 is a diagram showing a medical image and its file name according to the present invention, Figure 3 is a diagram showing a portion of a CSV file according to the present invention, and Figure 4 is a diagram showing the result of matching the medical image file name and CSV file according to the present invention. This is the drawing shown.

According to FIG. 2, the file name of the medical image may be an automatically generated file name. The file name may be randomly generated based on information on the time the image file was formed.

According to Figure 3, data for binary classification may include 1 and 0, where 1 may mean information that the image file is an image file of a dementia patient, and 0 may indicate that the image file is an image of a normal person. It may mean information called a file. 1 and 0 are just examples and can be interchanged or other numbers or letters can be used.

According to Figure 3, labeling through binary classification is defined based on the file name of the medical image. The above-described first data may refer to data defined by matching file names and labeling values, as shown in FIG. 3.

Figure 4 is a diagram showing a medical image classification and algorithm verification method according to the present invention.

According to FIG. 4, the present invention may include an image classification module 100 and a verification module 200. The medical image classification module 100 according to the present invention includes a dataset builder (Dataset Builder, 110) that matches image data and labeling data, and a CNN modeling module (ML) that performs machine learning (ML) on training images and verification images. 120), and an evaluation module 130 that performs performance evaluation using a test image.

The dataset builder 110 may be a module that can receive medical semantics (or eye image) and labeling data and match them. Additionally, the dataset builder 110 may divide the matched results into training images, validation images, and test images. Training images and verification images may be transmitted to the CNN modeling module 120, and test images may be transmitted to the evaluation module 130.

The dataset builder 110 may generate first data based on medical images and labeling data for determining dementia. The dataset builder 110 may generate the first data by matching the file name and labeling data of the medical image. That is, the dataset builder 110 can generate first data by matching the file name of the medical semantic paper and data for binary classification.

The CNN modeling module 120 may learn training images and verification images using a first learning algorithm. The CNN modeling module 120 may learn the medical image using a first learning algorithm and the first data. That is, the CNN modeling module 120 may generate an embedding value for the medical image and learn features of the medical image based on the embedding value and the first data. At this time, the first learning algorithm may be a learning algorithm using a plurality of layers according to the CNN learning algorithm.

In addition, the first learning algorithm includes a Siamese Network algorithm and a Logistic regression analysis algorithm, where the Siamese network includes a first network and a second network, and the first network is calculated using the embedding value. is replaced, the second network obtains a new embedding value for the new medical image, and the logistic regression analysis algorithm can perform binary classification based on the embedding value and the new embedding value. Additionally, the CNN modeling module 120 may modify the embedding value based on the new embedding value.

The evaluation module 130 may utilize a test image to evaluate the performance of the learned result. A test image for which the disease condition is known in advance is input to the image classification module 100, and the evaluation module 130 can obtain accuracy of the result. The learned results and the obtained accuracy may be transferred to and stored in the data storage unit 240. The evaluation module 130 can verify the results of the learning using a test image.

According to FIG. 4, the verification module 200 according to the present invention may include an automation module 210, an event module 220, and a result module 230. In addition, the verification module 200 according to the present invention may further include a data storage unit 240, or the result value may be transmitted to the data storage unit 240 and stored.

The automation module 210 may receive medical images (or eye images) that are unstructured data (Unstructured Date) and labeling data that are structured data (Structured Date). That is, unstructured data and structured data can be automatically input into the automation module 210. The automation module 210 can match unstructured data and structured data and transmit it to the event module 220.

A target image may be input to the event module 220. The event module 220 may extract embedding values for the target image and perform binary classification based on the extracted embedding values. As an example, the event module 220 may extract a distance based on the extracted embedding values, and classify it as 1 (classify it as True) if the target image is determined to be an eye image of a dementia patient. Additionally, the event module 220 may extract a distance based on the extracted embedding values, and classify it as 0 (classify it as False) if it is determined that the target image is not an eye image of a dementia patient.

The result module 230 may receive a result value classified as 1 and transmit it to the data storage unit 240. If classified as 0, the verification module 200 may transmit a message about the result.

According to Figure 4, the verification module that verifies the learning method according to the present invention may include a data input module, an event module, and a result module. The data input module can input unstructured data and structured data. That is, since the data input module includes a fundus camera, the above-mentioned medical images can be input, and since the data input module includes a computing device, the above-described CSV file can be input.

Figure 5 is a diagram showing a CNN network algorithm according to the present invention, Figure 6 is a diagram showing a CNN filter according to the present invention, and Figure 7 is a diagram showing a Siamese network algorithm according to the present invention.

According to Figures 5 and 6, the CNN (Convolutional Neural Network) algorithm is a widely used technique in the image processing field and can support automatic learning so that each element of the filter expressed as a matrix is suitable for data processing. Additionally, according to Figure 6, the types and functions of filters are shown.

In other words, the CNN algorithm may be a learning algorithm that uses multiple layers. In addition, the CNN algorithm can automatically learn filters that maximize image classification accuracy, and by adding new layers called convolutional layers and polling layers before the fully connected layer, the filtered image is obtained after applying the filtering technique to the original image. A classification operation can be performed on .

According to Figure 5, the CNN algorithm applies a filtering technique to the original image by adding a new layer called a convolutional layer and a pooling layer before the fully-connected layer, and then performs a classification operation on the filtered image. It can be configured to perform this.

At this time, the calculation formula for the image filter using the CNN algorithm is as shown in Equation 1 below.

[Equation 1]

(step, : Pixels in the ith row and jth column of the filtered image expressed as a matrix,

: filter,

: image,

: Height of filter (number of rows),

: Width of filter (number of columns))

According to FIG. 7, the Siamese network algorithm may be an algorithm for independently executing two CNN algorithms on two inputs. The Siamese network algorithm can run two CNN algorithms separately and compare the results. The Siamese network algorithm can obtain each feature map or embedding value as a result and obtain the distance value of each result. That is, the Siamese network may include a first network and a second network.

At this time, the distance value can be expressed as Equation 2 below.

[Equation 2]

(However, the distance d is defined as the norm between the vectors for x(1) and x(2))

In other words, two photos can be entered as input into the two CNN algorithms of the Siamese network and encoded. At this time, if the distance value of the two photos is less than the predetermined distance value, the two eye images have the same features. can see. In other words, if one eye image is labeled as 1 (i.e., corresponds to a dementia patient), and the distance between another eye image and the eye image labeled as 1 described above is less than a predetermined distance value, the other eye image also corresponds to a dementia patient. It can be interpreted as the eyeball of. To satisfy the conditions, continuous learning is necessary.

Preferably, the first learning algorithm of the present invention may include a Siamese Network algorithm and a Logistic regression analysis algorithm. At this time, the first network of the Siamese network can be replaced with the embedding value. That is, the first network does not execute the CNN algorithm, but can be replaced with embedding values for pre-extracted features. In the case of Siamese networks, resource consumption is high because two algorithms must be performed, but if one network is not executed and is replaced with embedding values for pre-extracted features, resource consumption can be reduced by nearly half. .

The second network of the Siamese network can obtain a new embedding value for a new medical image.

Additionally, the logistic regression analysis algorithm can binary classify new medical images into 1 and 0 based on the embedding value and the new embedding value. Additionally, the embedding value may be a value that is continuously modified based on the new embedding value.

Preferably, the calculation equation for the image filter using the CNN algorithm is as shown in Equation 3 below.

[Equation 3]

(step, : Pixel of the ith row and jth column of the filtered image expressed as a matrix,

: Application filter

: image,

: height of application filter (number of rows),

: Width (number of columns) of the application filter.

: patient age coefficient)

Preferably, is an application filter. If the patient's age is less than the first age, the conventional CNN filter can be used as is, and if the patient's age is greater than or equal to the first age, the patient age coefficient can be added to the conventional CNN filter. You can use the value multiplied by . At this time, Can be calculated by Equation 4 below.

[Equation 4]

Preferably, the first age is the age at which physical aging of the eye occurs according to statistics, and may be different for men and women. At this time, the patient age coefficient is a coefficient according to the age of the patient who is the subject of the eye image, and may mean a coefficient that changes depending on the patient's age. patient age coefficient The calculation formula for calculating is the same as Equation 5 below. (However, in the present invention, the patient's age is "full age", not Korean age.)

[Equation 5]

Preferably, in the present invention, the first age may be x years for men and x+3 years for women. As a result of multiple tests, the performance evaluation was the best when x was 40. In other words, as the patient ages, the thickness of the blood vessels or nerves in the eye may become thinner, so if we interpret the conclusion of the performance evaluation that performance improves, it can be interpreted that a little filter correction can increase the accuracy of eye image learning. there is.

Preferably, in the present invention, the second age may be higher than the first age. The second age may be y years for men and y+3 years for women. As a result of multiple tests, the performance evaluation was the best when y was 60. In other words, it can be interpreted that as the patient's aging progresses rapidly, the patient's age coefficient needs to change according to the aging rate, and through this, the accuracy of eye image learning can be increased.

The above-described present invention can be modeled as computer-readable code on a program-recorded medium. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. and also includes those modeled in the form of carrier waves (e.g., transmission over the Internet). Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of this specification should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of this specification are included in the scope of this specification.

Any or other embodiments of the present invention described above are not exclusive or distinct from each other. In certain embodiments or other embodiments of the present invention described above, each configuration or function may be used in combination or combined.

The above detailed description should not be construed as restrictive in any respect and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: Image classification module
200: Verification module

Claims (8)

In a medical image learning method for determining dementia, operated by a computer system including a processor,
generating, by the processor, first data based on medical images and labeling data for determining dementia;
learning, by the processor, the medical image using a first learning algorithm and the first data; and
Including, by the processor, verifying the results of the learning using a test image,
By the processor, the medical image is an eye image captured with a fundus camera,
In the step of generating the first data, the first data is generated by matching the file name of the medical image and the labeling data, and the file name of the medical image is randomly generated based on information on the time when the image file was formed,
The step of learning the medical image includes generating an embedding value for the medical image, learning features of the medical image based on the embedding value and the first data,
The first learning algorithm is a learning algorithm using a plurality of layers according to a CNN (Convolutional Neural Network) learning algorithm, and applies the patient age coefficient to the CNN filter of the CNN learning algorithm,
The first learning algorithm includes a Siamese Network algorithm, the Siamese network includes a first CNN network and a second CNN network, the first CNN network is replaced with the embedding value, and the second The CNN network obtains new embedding values for new medical images,
Two eye images are input to each of the first CNN network and the second CNN network of the Siamese network and encoding is performed, and if the distance value according to the two eye images is less than a predetermined distance value, the 2 A medical image learning method for dementia judgment that determines that a dog's eye image has the same characteristics and interprets it as the eye of a dementia patient. delete According to paragraph 1,
The labeling data includes data for binary classification,
The step of generating the first data is,
A medical image learning method for determining dementia, wherein the first data is generated by matching the file name of the medical image and the data for the binary classification. delete delete According to paragraph 1,
The first learning algorithm includes a logistic regression analysis algorithm,
The logistic regression analysis algorithm is,
A medical image learning method for determining dementia, which performs binary classification based on the embedding value and the new embedding value. According to clause 6,
A medical image learning method for determining dementia, wherein the embedding value is modified based on the new embedding value. A non-transitory computer-readable medium storing instructions, comprising:
The instructions, when executed by a processor, cause the processor to perform the method of claim 1, 3, 6, or 7.