KR102564750B1 - Deep learning algorithm for Alzheimer's disease prediction using a single brain MR slice - Google Patents

Abstract

The present invention relates to a deep learning method for predicting dementia risk based on a single brain MR slice image, comprising: (a) acquiring brain images for each patient by photographing brain shapes of a plurality of patients by a brain imaging unit; (b) receiving, by an error minimization unit, the obtained brain images, matching brain images for each patient, and generating error-minimized images of slice images through real-time data augmentation; (c) a deep learning module generating a weighted sum image by adding a weight in the learning step to the error-minimized image; and (d) predicting the risk of dementia by deriving, by a score definition module, a dementia index in the form of a real number for the generated summed images.

Description

Deep learning algorithm for Alzheimer's disease prediction using a single brain MR slice}

The present invention creates an error-minimized image of a slice image through matching brain images of a plurality of patients and real-time data augmentation, and uses a brain MR image T1 image to obtain one slice image from a viewpoint similar to human intuition. It is about a deep learning method for predicting dementia risk based on a single brain MR slice image that can derive a dementia index that is compatible for each hospital center and each medical equipment with only

- Fully convolution network (FCN):

Concept: It is a deep learning network composed of several convolution blocks (convolution layer, pooling layer), and it is characterized by the absence of a fully connected layer at the end.

In particular, since location information is not lost due to the nature of FCN, more accurate information can be obtained.

In the present invention, various models established in FCN are applied to derive results independent of hospital centers and medical equipment.

-Softmax layer:

Concept: In deep learning, the last activation kernel is designated. In the present invention, the softmax layer is used in the last stage because the goal is to derive it within the value range of the dementia index (0 to 1).

-Data augmentation:

Concept: When performing neural network modeling with large parameters in an environment where data is scarce, such as medical images, over-fitting occurs in the training data.

To solve this problem, there is also a method applied to the above content (FCN), and in the present invention, to dynamically increase data so that many images can be additionally generated from one slice image to be suitable for MR. use the technique

Through this, a more generalized dementia index is derived by reducing the effect of learning a large number with limited data and the bias of data by hospital center and medical equipment.

- Medical image processing :

Concept: Most brain T1 images use various medical image processing techniques/tools (freesurfer, SPM, FSL, etc.), but the core of the present invention is not to use medical image processing.

Derivation of the dementia index is possible only when appropriate slices are selected and input.

- Cortical thickness based ad risk score:

Concept: Cortical thickness is a term representing the thickness of the cerebral cortex of the brain. Since dementia causes brain tissue to be damaged, there is inevitably a difference in brain thickness between normal people and dementia patients.

Based on this, the proposed methodology is the ad risk score, which is expressed as a continuous number of normal (0) and dementia (1) according to the complex change in brain thickness.
In addition, as a prior art document related to the present invention described later, there is Republic of Korea No. 10-2019-0132832 (2019.11.29).

Therefore, in order to solve the above problems, the present invention has many existing techniques for determining the presence or absence of dementia based on MRI, but considering that most algorithms are not well applied when hospital / medical equipment is changed, The present invention aims to prove that the dementia index value comes out stably even when hospital/medical equipment is changed by imitating a process similar to human intuition.

When actual doctors read MR, they do not go through a complicated process like existing methodologies/tools (freesurfer, FSL, SPM, ...), but intuitively judge by looking at only a few areas. The present invention models and proves the process want to do

In addition, the purpose of matching brain images obtained for each patient to a standard brain template for each patient, generating images with minimized errors through real-time data augmentation, and generating weighted sum images by adding weights in the deep learning learning step there is

As a means for solving the above problems, the present invention includes (a) acquiring brain images for each patient by photographing the brain shapes of a plurality of patients by a brain imaging unit; (b) receiving, by an error minimization unit, the obtained brain images, matching brain images for each patient, and generating error-minimized images of slice images through real-time data augmentation; (c) a deep learning module generating a weighted sum image by adding a weight in the learning step to the error-minimized image; and (d) predicting the risk of dementia by deriving, by the score definition module, a dementia index in the form of a real number for the generated summed images.

As a means for solving the above problems, the present invention provides that the step (b) includes: (b-1) matching the brain images for each patient with a standard brain template by a data correction unit receiving the acquired brain images; (b-2) selecting a slice position by a slice position selection unit receiving the matched brain image for each patient; and (b-3) generating, by a data augmentation module, the error-minimized image of the slice image by applying data augmentation in real time to the location-selected slice; It is characterized in that it includes.

As a means for solving the above problems, the present invention provides that the step (b-2) includes manually selecting a slice including a location related to dementia; and automatically selecting a slice including the center of gravity of the hippocampus in the coronal direction and a slice including the center of gravity of the ventricle in the axial direction.

As a means for solving the above problems, the present invention, in the step (b-3), the parameter used for data augmentation is one of horizontal and vertical symmetry, scaling, tilting, rotation, normalization, movement, and brightness control. It is characterized by more than

As a means for solving the above problems, the present invention is characterized in that the step (a) uses a non-linear matching method and a linear matching method.

As a means for solving the above problems, the present invention is characterized in that in step (d), a logistic regression model is applied when there is one category, and softmax is applied when there are two or more categories.

Details of other embodiments are included in "Specific Contents for Carrying Out the Invention" and the accompanying "Drawings".

Advantages and/or features of the present invention, and methods of achieving them, will become apparent upon reference to the various embodiments described below in detail in conjunction with the accompanying drawings.

However, the present invention is not limited only to the configuration of each embodiment disclosed below, but may also be implemented in various other forms, and each embodiment disclosed herein only makes the disclosure of the present invention complete, and this It is provided to completely inform the scope of the present invention to those skilled in the art to which the invention belongs, and it should be noted that the present invention is only defined by the scope of each claim of the claims.

The present invention takes a long time in the existing computing methodology and is different from intuition, so it is not compatible for each hospital/medical equipment, but it has the advantage of being able to automatically proceed with all processes, and the inconvenience of having to do it manually when a person reads it. However, it takes advantage of not being limited to hospital/medical equipment, and if only modeling is done, a time of several minutes is sufficient to derive the dementia index.

In addition, the brain image of each patient is modified similarly to a standard brain template, and the slice location is selected based on the area where the error between multiple hospital centers and between multiple medical devices is minimized. It is possible to minimize errors caused by different shapes.

In addition, in the deep learning step, weights are added to the normal image and the dementia image, thereby minimizing an error for extreme classification into a normal state or a dementia state.

1 is a block diagram of an apparatus for implementing a deep learning method for predicting dementia risk based on a single brain MR slice image according to an embodiment of the present invention.
2 is a flowchart illustrating the operation of a deep learning method for predicting risk of dementia based on a single brain MR slice image according to an embodiment of the present invention.
FIG. 3 is a flowchart for explaining the detailed operation of step S200 during the operation of the deep learning method for predicting dementia risk shown in FIG. 2 .
4 is a diagram showing a graph of a test result using a dementia risk prediction deep learning method according to an embodiment of the present invention, compared with an existing methodology.

The following merely illustrates the principles of the present invention. Therefore, those skilled in the art can invent various devices that embody the principles of the present invention and fall within the concept and scope of the present invention, even though not explicitly described or shown herein. In addition, it is to be understood that all conditional terms and embodiments listed herein are, in principle, expressly intended only for the purpose of making the concept of the present invention understood, and not limited to such specifically listed embodiments and conditions. It should be.

Further, it should be understood that all detailed descriptions reciting specific embodiments, as well as principles, aspects and embodiments of the present invention, are intended to encompass structural and functional equivalents of these matters. In addition, it should be understood that such equivalents include not only currently known equivalents but also equivalents developed in the future, that is, all devices invented to perform the same function regardless of structure.

Thus, for example, the block diagrams herein should be understood to represent conceptual views of illustrative circuits embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc., are meant to be tangibly represented on computer readable media and represent various processes performed by a computer or processor, whether or not the computer or processor is explicitly depicted. It should be.

The functions of the various elements shown in the drawings including functional blocks represented by processors or similar concepts may be provided using dedicated hardware as well as hardware capable of executing software in conjunction with appropriate software.

When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, the explicit use of terms presented as processor, control, or similar concepts should not be construed as exclusively citing hardware capable of executing software, but without limitation, digital signal processor (DSP) hardware, ROM for storing software (ROM), random access memory (RAM) and non-volatile memory. Other hardware for the governor's use may also be included.

In the claims of this specification, components expressed as means for performing the functions described in the detailed description include, for example, a combination of circuit elements performing the functions or all types of software including firmware/microcode, etc. It is intended to include any method that performs the function of performing the function, combined with suitable circuitry for executing the software to perform the function.

Since the invention defined by these claims combines the functions provided by the various enumerated means and is combined in the manner required by the claims, any means capable of providing such functions is equivalent to that discerned from this specification. should be understood as

The above objects, features and advantages will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily implement the technical idea of the present invention. There will be.

In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

1 is a block diagram of an apparatus for implementing a deep learning method for predicting dementia risk based on a single brain MR slice image according to an embodiment of the present invention, including a brain image capture unit 100, an error minimization unit 200, and deep learning module 300 and score definition module 400 .

The error minimization unit 200 includes a data correction unit 210 , a slice location selection unit 220 and a data augmentation module 230 .

2 is a flowchart illustrating the operation of a deep learning method for predicting risk of dementia based on a single brain MR slice image according to an embodiment of the present invention.

FIG. 3 is a flowchart for explaining the detailed operation of step S200 during the operation of the deep learning method for predicting dementia risk shown in FIG. 2 .

The operation of the deep learning method for predicting dementia risk based on a single brain MR slice image according to an embodiment of the present invention is schematically described with reference to FIGS. 1 to 3 as follows.

First, the brain imaging unit 100 acquires brain images for each patient by capturing the brain shapes of a plurality of patients (S100).

The error minimization unit 200 receives the brain image obtained from the brain imaging unit 100, matches the brain images for each patient, and generates an error-minimized image of the slice image through real-time data augmentation (S200). .

That is, when the data correction unit 210 receives the brain image obtained from the brain imaging unit 100 and matches the brain image for each patient to the standard brain template (S210), the slice position selector 220 selects the data The correction unit 210 receives the matching brain image for each patient and selects a slice position (S220).

In addition, the data augmentation module 230 generates an error-minimized image of the slice image by applying data augmentation in real time to the slice location selected by the slice location selector 220 (S230).

The deep learning module 300 generates a weighted summed image by adding weights in the learning step to the error-minimized image in step S200 (S300).

The score definition module 400 predicts the risk of dementia by deriving a dementia index in the form of a real number for the summed images generated in step S300 (S400).

The organic operation of the deep learning method for predicting risk of dementia based on a single brain MR slice image according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 3 .

The present invention enables an individual to derive a compatible dementia index for each hospital center and medical equipment from brain images taken after visiting medical institutions ranging from large university hospitals to clinics.

In particular, the present invention can be implemented using only minimized information (one slice) without the need to supplement the results through the use of neuropsychological tests.

The brain imaging unit 100 acquires brain images for each patient by photographing the brain shapes of a plurality of patients.

The range of 'obtained brain images' of the present invention includes brain image slices in which the structure can be judged by the intuition of the MR medical staff.

That is, it includes most of the T1 MRs from high-resolution MRs (eg, 200 or more z-axis MRs) to low-resolution MRs (eg, about 20 z-axis MRs).

The data correction unit 210 receives the brain image obtained for each patient and matches the brain image for each patient to a standard brain template using nonlinear and linear matching methods.

Through this, each patient's brain image is deformed similarly to a standard brain template, and most of the tissues are positioned similarly, thereby minimizing errors caused by differences in brain shapes between patients (error minimization step 1).

The slice location selection unit 220 selects a slice location based on a region based on a location where an error between multiple hospital centers and between multiple medical devices is minimized through many experiments (error minimization step 2).

That is, a slice containing dementia-related locations (gray matter, hippocampus, etc.) is manually selected and used as an input.

In addition, the slice containing the center of gravity of the hippocampus in the coronal plane direction and the slice containing the center of gravity of the ventricle in the axial direction are automatically selected and used as input for the next part.

The data augmentation module applies data augmentation because there is a limitation with a small amount of data in the case of medical images. Through random generation such as movement and brightness control, images are generated infinitely for slice images, and through this, a compatible dementia index different from other onsets can be reached (error minimization step 3).

In addition, in addition to data augmentation that transforms the image, the deep learning module 300 adds weight to the normal image and the dementia image in the learning step to generate a weighted sum image at each learning time By doing so, the error for being extremely classified as a normal state or a dementia state is minimized (error minimization step 4).

The deep learning module 300 can use most of the deep learning network modules, but confirms that the use of the network to which the inception module is applied is suitable for deriving a compatible dementia index.

As a result of conducting multiple experiments, it was confirmed that the inception module-based deep learning network learns by looking at patterns similar to human perception, rather than simply image features.

The score definition module 400 is a logistic regression model if there is one category so that the dementia index can be derived in a continuous (real form) rather than a classification value of discontinuous values such as {normal, cognitive mildness, dementia} , and when there are many categories, that is, when there are two or more categories, softmax is applied to predict the risk of dementia by deriving values in continuous (real number form) scores.

The closer to normal, the closer to 0, and the closer to dementia, closer to 1, and the range is not limited to [0, 1].

If a cut-off is specified in the form of continuous real numbers, prediction can be specified with a classification value as in other techniques.

4 is a diagram showing a graph of a test result using a dementia risk prediction deep learning method according to an embodiment of the present invention, compared with an existing methodology.

In this embodiment, the experiment was conducted using the brain MR image dataset (T1) obtained from Samsung Seoul Hospital, ADNI, Gachon University Hospital, and Chaum in the United States, and about 94% of the results were obtained for normal/dementia classification.

As a result of testing two hospital centers and medical equipment for the same person, it was confirmed that each hospital center and medical equipment converged closer to the diagonal compared to the existing methodology.

In this way, the present invention imitates a process similar to human intuition, so that the dementia index numerical value is stably calculated even when hospital centers/medical equipment are changed, matching brain images obtained for each patient to a standard brain template for each patient, , Provides a single-brain MR slice image-based dementia risk prediction deep learning method that generates an error-minimized image through real-time data augmentation and generates a weighted summed image by adding weights in the deep learning learning step.

Through this, it is possible to be compatible with each hospital center/medical equipment, all processes are automatically performed, and only a few minutes are sufficient to derive the dementia index if only modeling is performed.

In addition, each patient's brain image is deformed similarly to a standard brain template, and the brain shape between each patient is selected based on the area where the error between multi-hospital centers and multi-medical equipment is minimized. Errors caused by this difference can be minimized.

In addition, the slice position is selected based on the region where the error between multi-hospital centers and multi-medical equipment is minimized, and weights are added to normal images and dementia images in the deep learning step to return to a normal or dementia state. It is possible to minimize the error for being classified as extreme.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and is common in the art to which the present invention pertains without departing from the gist of the present invention claimed in the claims. Of course, various modifications and implementations are possible by those with knowledge of, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: brain imaging unit
200: error minimization unit
210: data correction unit
220: slice position selection unit
230: data augmentation module
300: deep learning module
400: score definition module

Claims (6)

(a) acquiring brain images for each patient by photographing the brain shapes of a plurality of patients, by a brain imaging unit of the dementia risk prediction deep learning device;
(b) The error minimization unit of the dementia risk prediction deep learning device receives the obtained brain image and matches the brain image for each patient with a standard brain template, and derives the error between multi-hospital centers and multi-medical equipment to be minimized The slice position is selected based on the area to be selected, and the error minimized image for the slice image through real-time data augmentation using parameters of left and right and top and bottom symmetry, scaling, tilting, rotation, normalization, movement, and brightness control generating;
(c) The deep learning module of the dementia risk prediction deep learning device generates a weighted sum image by adding a weight in the learning step to the error-minimized image, and adds weight to the normal image and the dementia image generating a weighted sum image at each training time; and
(d) predicting, by the score definition module of the dementia risk prediction deep learning device, a dementia risk by deriving a dementia index in the form of a real number for the generated sum images;
Characterized in that it includes,
A Deep Learning Method for Dementia Risk Prediction Based on Single Brain MR Slice Images.
According to claim 1,
The step (b) is
(b-1) receiving the obtained brain image and matching the brain image for each patient with a standard brain template, by a data correction unit of the dementia risk prediction deep learning device;
(b-2) selecting a slice position by receiving the matched brain image for each patient by a slice position selector of the dementia risk prediction deep learning device; and
(b-3) generating, by the data augmentation module of the dementia risk prediction deep learning apparatus, the error-minimized image of the slice image by applying data augmentation in real time to the location-selected slice;
Characterized in that it includes,
A Deep Learning Method for Dementia Risk Prediction Based on Single Brain MR Slice Images.
According to claim 2,
The step (b-2) is
Automatically selecting, by the slice position selection unit of the dementia risk prediction deep learning device, a slice including the center of gravity of the hippocampus in the coronal direction and a slice including the center of gravity of the ventricle in the axial direction;
Characterized in that it includes,
A Deep Learning Method for Dementia Risk Prediction Based on Single Brain MR Slice Images.
delete According to claim 1,
The step (a) is
A deep learning method for predicting dementia risk based on a single brain MR slice image, characterized by using nonlinear matching and linear matching methods.
According to claim 1,
In step (d),
A deep learning method for predicting dementia risk based on a single brain MR slice image, characterized by applying a logistic regression model when there is one category and applying softmax when there are two or more categories.