KR20230024446A - Acute ischemic stroke diagnosis apparatus and method using deep-learning model based on 3-d cnn - Google Patents

Abstract

According to the present invention, a method for diagnosing an acute ischemic stroke using a deep learning model based on 3D-CNN, the method comprising the steps of: collecting 3D volume data including a diffusion weighted image (DWI) and an apparent diffusion coefficient (ADC) map as data generated by photographing the brain of a patient; dividing and outputting a lesion area related to an acute ischemic stroke in the 3D volume data by using a trained lesion segmentation model as a predefined deep learning model; and classifying and outputting a subtype corresponding to a cause mechanism of an acute ischemic stroke based on the 3D volume data and the divided lesion area by using a trained subtype classification model as the predefined deep learning model. Accordingly, the present invention can classify subtypes associated with cause mechanisms with high accuracy.

Description

Apparatus and method for diagnosing acute ischemic stroke using 3D-CNN-based deep learning model

The present invention relates to a technology for diagnosing acute ischemic stroke, capable of segmenting lesions of acute ischemic stroke and classifying subtypes using a 3D-CNN-based deep learning model.

Acute ischemic stroke is a disease with several causes. Therefore, classifying the mechanism of acute ischemic stroke is essential for treatment and secondary prevention. Although TOAST classification is currently the most widely used system, it often has limitations in classifying unknown causes and also has limitations in terms of inter-rater reliability.

To overcome these limitations, various deep learning algorithms based on convolutional neural networks (CNNs) have been proposed as techniques for diagnosing acute ischemic stroke in brain MRI images. However, these conventional studies have limited to segmentation of stroke lesions, and no technique has been proposed for predicting the mechanism of occurrence of stroke and classifying subtypes associated therewith.

The present applicant has come to the present invention by recognizing the limitations of the prior art and conducting research that can help prognosis treatment and secondary prevention by predicting a more accurate mechanism of occurrence for patients with acute ischemic stroke.

Korean Patent Registration No. 10-2015224

The present invention was derived to solve the above problems, and aims to provide a diagnostic technique capable of segmenting lesions of acute ischemic stroke and classifying subtypes using a 3D-CNN-based deep learning model.

According to one aspect of the present invention, a method for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model is data generated by photographing a patient's brain, and diffusion weighted image (DWI) and apparent diffusion Collecting 3D volume data including an Apparent Diffusion Coefficient (ADC) map; segmenting and outputting a lesion region associated with an acute ischemic stroke from the 3D volume data using a lesion segmentation model for which learning has been completed as a predefined deep learning model; and classifying and outputting a subtype corresponding to a causal mechanism of acute ischemic stroke based on the 3D volume data and the segmented lesion area using a subtype classification model for which learning has been completed as a predefined deep learning model; includes

In one embodiment, the lesion segmentation model includes the DWI and ADC maps defined as input data, an encoder extracting a feature map from the input data, and a decoder predicting a lesion area using the extracted feature map. can be configured.

In one embodiment, the subtype classification model defines the DWI and ADC maps as input data, and uses the lesion area divided by the lesion segmentation model to derive a focus when subtype prediction is used as a feature map. It may include an attention mechanism that strengthens.

In one embodiment, the subtype classification model may predict a probability for a subtype including at least one of large artery atherosclerosis (LAA), cardiac embolism (CE), and small vessel occlusion (SVO).

In an embodiment, a residual block for feature extraction may be applied to the lesion segmentation model and the subtype classification model.

According to another aspect of the present invention, an apparatus for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model includes 3D volume data including DWI and ADC maps as data generated by photographing a patient's brain. Data collection unit to collect; a lesion segmentation unit that divides and outputs a lesion region associated with an acute ischemic stroke from the 3D volume data using a lesion segmentation model that has been learned as a predefined deep learning model; and a subtype that classifies and outputs a subtype corresponding to a causal mechanism of acute ischemic stroke based on the 3D volume data and the segmented lesion area using a subtype classification model for which learning has been completed as a predefined deep learning model. Classification unit; includes.

The present invention can accurately predict and segment even very small lesions through a lesion segmentation model and subtype classification model learned using DWI and ADC of acute ischemic stroke patients as input data, and subtypes associated with causal mechanisms can be classified with high accuracy can be classified as

1 is a reference diagram showing prediction results by a lesion segmentation model according to the present invention.
2 is a reference diagram showing a failure case of the lesion segmentation model according to the present invention.
3 is a confusion matrix for the subtype classification model according to the present invention.
4 is a reference diagram for explaining a data population according to the present invention.
5 is a reference diagram for explaining a lesion segmentation model according to the present invention.
6 is a reference diagram for explaining a subtype classification model according to the present invention.
7 is a block diagram illustrating an apparatus for diagnosing acute ischemic stroke according to the present invention.
8 is a flowchart illustrating a method for diagnosing acute ischemic stroke according to the present invention.

Since the present invention can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it should be understood that this is not intended to limit the present invention to specific embodiments, and includes all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1. Introduction

Acute ischemic stroke has various causes depending on the causative mechanism, and includes large artery atherosclerosis (LAA), cardioembolism (CE), small vessel occlusion (SVO), and others. It can be classified as a stroke of other determined etiology or a stroke of undetermined etiology. Classification of acute ischemic stroke according to cause is important for treatment and secondary prevention. The most widely used classification system is the Trial of Org 10172 in Acute Stroke Treatment (TOAST) classification. However, these classification methods have limitations, such as showing a moderate level of inter-rater reliability in classifying acute ischemic stroke, and often classifying stroke as an unknown pathology. Currently, the development of technology for overcoming these limitations is in progress, but the situation has not shown sufficient results.

In order to determine the causal mechanism of acute ischemic stroke, various diagnostic methods such as brain imaging and cardiac examination are required. Early diagnosis of stroke subclasses through this classification system can have a positive impact on patient treatment, prognosis, and secondary prevention. Diffusion Weighted MR Imaging (MR Imaging) is a method widely used for acute stroke diagnosis. It shows excellent performance in detecting hyperacute lesions and very small ischemic lesions. Excellent discrimination performance. In addition, when ADC map and DWI are used simultaneously, acute ischemic stroke lesions can be more accurately distinguished and important information regarding the appropriate treatment timing for lesions can be provided. It has been reported that diffuse imaging lesion patterns, which provide useful information for early diagnosis of acute ischemic stroke, are closely related to stroke subtypes.

As a technique for diagnosing acute ischemic stroke in brain MRI images, various deep learning algorithms based on convolutional neural networks (CNNs) have been proposed. These previous studies have shown that deep learning techniques can more accurately detect stroke lesions than conventional machine learning techniques and can extract meaningful features for severity evaluation or prognosis prediction. Some researchers have proposed a lesion segmentation technology based on the U-Net architecture for patients with acute ischemic stroke. To efficiently utilize the context information of volumetric MRI data, Zhang et al. proposed a stroke lesion segmentation technique using a 3-D fully connected-DenseNet (DenseNet). The above studies have demonstrated that deep learning technology can classify acute ischemic stroke patients through lesion segmentation, but a classification technique for predicting the treatment mechanism of acute ischemic stroke has not yet been proposed. The present applicant has devised the present invention related to a 3D CNN-based deep learning model that enables stroke lesion segmentation and subtype classification using only DWI and ADC images of patients with acute ischemic stroke, and the present invention will be described in detail below. do.

2. Results

(1) lesion segmentation model

The lesion segmentation model according to the present invention showed a Dice score of 0.891±0.034 in the training set. In the case of the test set, the lesion segmentation model according to the present invention showed Dice scores, precision, and recall of 0.843±0.009, 0.842±0.012, and 0.844±0.017, respectively. 1 is a reference diagram showing prediction results by a lesion segmentation model according to the present invention (in FIG. 1, the first row is DWI, the second row is ADC, and the third row is GT annotated by a neurologist). (ground truth, correct answer value) labels, and the fourth row is the lesion area predicted by the lesion segmentation model according to the present invention). Referring to FIG. 1 , it can be seen that the lesion segmentation model according to the present invention can accurately predict very small lesions. Meanwhile, FIG. 2 is a reference diagram showing a failure case of the lesion segmentation model according to the present invention. Referring to FIG. 2 , it can be seen that most failure cases occur when a lesion has very poor contrast in a diffusion image.

(2) Stroke subtype classification model

The test set average accuracy of the stroke subtype classification model according to the present invention was 81.9%. 3 is a confusion matrix for the stroke subtype classification model according to the present invention. Referring to FIG. 3, the stroke subtype classification model according to the present invention showed lower accuracy of SVO than other subtypes, which is because the classification model according to the present invention has relatively low analysis performance for small lesions, As a control, it indicates some confusion.

3. Discussion

The present invention is the first technology to classify subtypes of stroke mechanisms by analyzing acute ischemic stroke lesion patterns through deep learning based on 3D-CNN using DWI and ADC of patients with acute ischemic stroke. The main results according to the present invention are as follows. First, the 3D-CNN-based segmentation accuracy for acute ischemic stroke lesions was found to be 0.843 in the Dyce score. Second, in terms of subtype classification to classify causes of acute ischemic stroke, the predictive value of cause classification according to TOAST classification was 81.3% for LAA, 84.6% for SVO, and 73.0% for CE, respectively.

Brain imaging plays an important role in diagnosing and identifying the mechanisms that cause acute ischemic stroke in line with technological advances. Among various MR sequences, DWI and ADC maps are useful tools for early detection of acute ischemic lesions and for distinguishing between stroke mimics and acute ischemic stroke. Various previous studies have attempted to segment the infarction volume of acute ischemic stroke on MRI using artificial intelligence. It is known that various imaging patterns of acute ischemic stroke in DWI lesions correlate with pathogenic mechanisms. In the case of a cardiac embolic stroke, acute stroke lesions in DWI often show a single cortical or subcortical lesion or occur multiple times in multiple vascular branches, which may affect the anterior circulation. Multiple unilateral lesions are a characteristic finding of arterial embolism. On the other hand, one small infarct (2-20 mm in diameter) lesion observed in the deep acute ischemic stroke white matter, basal ganglia, thalamus, and pons results in small vessel occlusion. It has a high correlation with small vessel occlusion. In the present invention, it is intended to apply a doctor's diagnosis process to identify the cause of acute ischemic stroke based on the characteristic findings of brain MRI using artificial intelligence.

However, there are many limitations in predicting the pathogenesis of acute ischemic stroke using only DWI and ADC maps. The TOAST classification is the most widely used system for classifying acute ischemic stroke according to pathogenesis. Clinical findings and findings from ancillary diagnostic studies, including brain imaging and cardiac evaluation, are used to classify a patient's mechanism of acute ischemic stroke. However, despite ease of use and widespread popularity, the overall inter-rater agreement is moderate, and in particular, the reliability of small vessel occlusion and stroke of unknown origin compared to large artery atherosclerosis or cardiac embolism is poor. known to be low. To overcome this, an improved classification method applying a new diagnostic technique was proposed, but it still showed limitations. In particular, acute ischemic stroke caused by an unknown mechanism other than the mechanism that can be explained, such as large artery atherosclerosis, cardiac embolism, small vessel occlusion, and stroke due to other causes, is referred to as a cryptogenic stroke. It is observed in some patients with acute ischemic stroke. These strokes of unknown cause are often observed as embolic strokes, and are called ESUS (embolic strokes of undetermined source). There is a need to clarify the mechanism of ESUS and provide appropriate treatment, but there is no clear method yet. The present applicant devised the present invention by determining that diagnosis of acute ischemic stroke using a deep learning algorithm would be an answer. Applicants anticipate that the present invention can be applied to ESUS patients as a useful tool for distinguishing between cardiac embolic stroke and non-cardioembolic stroke.

In addition, the present invention shows a clear difference from previous studies in terms of technology. Conventional lesion segmentation techniques use a single slide as an input and show low accuracy because they cannot utilize information between adjacent slides. A lesion segmentation technique based on 3D CNN has been proposed, but its analysis performance is limited due to its relatively shallow network architecture. On the other hand, the lesion segmentation technique according to the present invention showed a high Dice score (0.845) in the test set as a residual network was applied to allow stable model learning even in a deep network architecture. In addition, the subtype classification model according to the present invention showed an average classification accuracy of 81.9% through an attention mechanism using lesion information predicted in the lesion segmentation model. Unlike the prior art, the present invention may provide a focus area for performance improvement of the classification model in addition to the subtypes of stroke. The segmentation and classification model according to the present invention can be easily applied to various medical AI fields that use 3D volumes as input information, such as CT and MRI.

The present applicant has devised the present invention by conducting research that can help prognosis treatment and secondary prevention by more accurately predicting the mechanism of occurrence of patients with acute ischemic stroke. The present invention relates to a 3D-CNN-based deep learning algorithm using only initial MRI information, and presents a feasible model for predicting the mechanism of occurrence. Diffuse lesion volume measurement and stroke subtype classification according to the present invention showed a strong correlation with those performed by neurologists' manual segmentation and subtype classification. The present invention is of great significance as the first technology to predict the occurrence of acute ischemic stroke using only MRI information.

4. Deep learning model

(1) Study population

In order to derive the present invention, information of 2,251 acute ischemic stroke patients (patients who visited Chungbuk National University Hospital from February 2013 to July 2019) was utilized. Information on acute ischemic stroke was compiled using a registry. All patients with acute ischemic stroke were reviewed by at least three stroke specialists and classified according to the TOAST classification. There were a total of 1,768 patients with large artery atherosclerosis, cardiac embolism, and small vessel occlusion (excluding strokes due to other causes or strokes of unknown cause) according to the TOAST classification. Among them, 1,408 patients had DWI and ADC (see Fig. 4). There were 608 patients with large artery atherosclerosis, 441 patients with cardiac embolism, and 359 patients with small vessel occlusion. Among the normal patients who visited the hospital during the same period, 400 patients with normal MRI findings were included as controls. The patient's gender, age, and NIHSS (National Institutes of Health Stroke Scale Rating) are shown in Table 1 below.

[Table 1]

(2) image acquisition

MRI was acquired through various devices including 1.5T (Achieva, Philips Healthcare) and 3.0T (Ingenia CX, Philips Healthcare; Acquetva, Philips Healthcare) scanners. The parameters of the DWI sequence are as follows. Repetition time 2500-3000ms, echo time 80ms, slice thickness 3-5mm, intersection gap 1mm, field of view (FOV) 220x220mm, matrix size (approx. 2mm x 2mm flat resolution) 256x256, b-values 0 and 1000 s/mm2. (Each ADC map was automatically generated by the manufacturer's software)

(3) data annotation

To create a ground truth (GT) reference standard for training and evaluating subtype classification models, each patient was classified into four types (LAA, SVO, CE, Control) according to the TOAST classification system. Afterwards, the lesion area on each DWI slide was manually annotated (using self-developed software) by two neurologists with high expertise. Thereafter, labeling for each lesion was completed with cross-validation and final decision by two raters.

(4) lesion segmentation model

The lesion segmentation model according to the present invention is based on a 3D CNN called V-Net (Fully convolutional neural networks for volumetric medical image segmentation, 565-571, IEEE, 2016). 5 is a reference diagram for explaining a lesion segmentation model according to the present invention. Referring to FIG. 5, the lesion segmentation model according to the present invention includes an encoder (left) extracting feature maps from a local 3D volume and a decoder (right) predicting stroke lesions using the feature maps. It consists of Since the lesion segmentation model according to the present invention has a very deep architecture, a residual block is applied to solve the learning instability caused by the gradient vanishing and exploding problem. Here, the residual block consists of 1) a 3D convolution layer with a kernel size of 3x3x3, 2) a residual skip connection, and two 3D convolution layers (kernel size 3x3x3). In the network encoder, the residual block is used for feature extraction and max-pooling layers, and a stride of 2 is applied to reduce the spectral dimensionality. On the other hand, the decoder is composed of an up-convolution layer (with a stride of 2) followed by a residual block after feature map concatenation. Skipping connections from encoder-side layers with the same resolution provide high-resolution features to the decoder side. In the last layer of the decoder, a sigmoid activation layer is concatenated to compute a probability map for stroke lesions.

The lesion segmentation model according to the present invention was trained for more than 200 epochs with the Adam Optimizer, initial learning rate 1e-5, and batch size set 8, and was trained from scratch without using pre-trained weights . Various loss functions (weighted cross-entropy loss, L1 loss, and Dice loss) were tested for the lesion segmentation model according to the present invention, and Dice loss showed the best performance. In addition, various data augmentation techniques (rigid transformation, horizontal/vertical flip, Gaussian noise, and gamma correction) were randomly triggered in each training session to address the lack of data.

(5) subtype classification model

은 아래 수학식 1에 따라 얻어질 수 있다.6 is a reference diagram for explaining a subtype classification model according to the present invention. Referring to FIG. 6 , the subtype classification model according to the present invention predicts four types of probabilities composed of LAA, SVO, CE, and MATCH as a control. In the subtype classification model according to the present invention, residual blocks of the above-described lesion segmentation model are applied for feature extraction, and lesion prediction provided by the lesion segmentation model is used to guide network focus on lesions. The result is used to enhance the feature map. Here, the enhanced feature map

Can be obtained according to Equation 1 below.

[Equation 1]

In Equation 1, F is a feature map extracted by the residual block, and A is a lesion segmentation result. h(A) is a bilinear interpolation that matches the spatial resolution between F and A. This attention mechanism induces the network to focus on the lesion area to predict the stroke subtype, thereby significantly improving the classification performance of the model. The subtype classification model according to the present invention was trained over 400 epochs with Adam Optimizer, an initial learning rate of 1e-5, and a batch size set of 4, and was trained from scratch. Meanwhile, categorical cross-entropy loss was applied and data augmentation techniques were used during training.

5. Method and apparatus for diagnosing acute ischemic stroke according to the present invention

A method for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model (hereinafter, a diagnosis method) according to the present invention is performed in an apparatus for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model (hereinafter, a diagnosis apparatus) .

7 is a block diagram illustrating a diagnosis device 700 according to the present invention, and FIG. 8 is a flowchart illustrating a diagnosis method performed by the diagnosis device 700. Hereinafter, a diagnosis apparatus and method according to the present invention will be described with reference to FIGS. 7 and 8 , but details overlapping with those described above will be omitted.

Referring to FIG. 7 , the diagnosis apparatus 700 according to the present invention includes a data collection unit 710, a lesion segmentation unit 720, and a sub-type classifying unit 730, each of which will be described below. It may correspond to a computing device in which an operation is implemented.

The data collection unit 710 collects 3D volume data including a diffusion weighted image (DWI) and an apparent diffusion coefficient (ADC) map as data generated by photographing the patient's brain (step S810). Meanwhile, the data collection unit 710 may collect basic data (DWI, ADC map) for learning the deep learning model according to the present invention as well as data (DWI, ADC map) for diagnosis.

The lesion segmentation unit 720 segments the lesion area associated with the acute ischemic stroke from the 3D volume data using the trained lesion segmentation model and outputs (see FIG. 5, segmentation results) (step S820). As described in 4. (4) above, the lesion segmentation model can be defined with DWI and ADC maps as input data, and an encoder that extracts a feature map from the input data and predicts a lesion area using the extracted feature map. It may correspond to a deep learning model of a V-Net structure including a decoder. In addition, a residual block for feature extraction may be applied to the lesion segmentation model.

The subtype classification unit 730 classifies and outputs a subtype corresponding to the causal mechanism of acute ischemic stroke based on the 3D volume data and the segmented lesion area using the subtype classification model that has been learned (step S830). . As described in 4. (5) above, the subtype classification model can be defined with DWI and ADC maps as input data, and based on the lesion area segmented by the lesion segmentation model to induce concentration in subtype prediction. It may include an attention mechanism that enhances the feature map with . Also, a residual block for feature extraction may be applied to the subtype classification model. The subtype classification unit 730 predicts the probability of a subtype including at least one of large artery atherosclerosis (LAA), cardiac embolism (CE), and small vessel occlusion (SVO) (see FIG. 6, Probability for each subtype). Here, the subtype classification unit 730 may determine and output the subtype having the highest probability as a subtype corresponding to the cause mechanism of acute ischemic stroke.

The method for diagnosing acute ischemic stroke using the 3D-CNN-based deep learning model according to the present invention described above can be implemented as computer readable code on a computer readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium is distributed in computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner.

The preferred embodiments of the present invention described above have been disclosed for illustrative purposes, and those skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and these modifications, changes, and The additions should be viewed as falling within the scope of the following claims.

700: Acute ischemic stroke diagnostic device
710: data collection unit
720: lesion division
730: subtype classification unit

Claims (6)

In the method for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model,
Collecting 3D volume data including diffusion weighted image (DWI) and apparent diffusion coefficient (ADC) maps as data generated by photographing the patient's brain;
segmenting and outputting a lesion region associated with an acute ischemic stroke from the 3D volume data using a lesion segmentation model for which learning has been completed as a predefined deep learning model; and
Classifying and outputting a subtype corresponding to a causal mechanism of acute ischemic stroke based on the 3D volume data and the divided lesion area using a subtype classification model for which learning has been completed as a predefined deep learning model; including
A method for diagnosing acute ischemic stroke using a deep learning model based on 3D-CNN.
The method of claim 1, wherein the lesion segmentation model,
The DWI and ADC maps are defined as input data, and the deep learning of the V-NET structure includes an encoder that extracts a feature map from the input data and a decoder that predicts a lesion area using the extracted feature map. characterized by being a model,
A method for diagnosing acute ischemic stroke using a deep learning model based on 3D-CNN.
The method of claim 1, wherein the subtype classification model,
The DWI and ADC maps are defined as input data,
In order to induce attention when predicting subtypes, it is characterized by including an attention mechanism structure for strengthening a feature map based on the lesion area segmented by the lesion segmentation model.
A method for diagnosing acute ischemic stroke using a deep learning model based on 3D-CNN.
The method of claim 3, wherein the subtype classification model,
Characterized in that predicting the probability for a subtype comprising at least one of large artery atherosclerosis (LAA), cardiac embolism (CE) and small vessel occlusion (SVO),
A method for diagnosing acute ischemic stroke using a deep learning model based on 3D-CNN.
The method of claim 3, wherein the lesion segmentation model and the subtype classification model,
Characterized in that a residual block for feature extraction is applied,
A method for diagnosing acute ischemic stroke using a deep learning model based on 3D-CNN.
A device for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model,
A data collection unit for collecting 3D volume data including diffusion weighted image (DWI) and apparent diffusion coefficient (ADC) maps as data generated by photographing the patient's brain;
a lesion segmentation unit that divides and outputs a lesion region associated with an acute ischemic stroke from the 3D volume data using a lesion segmentation model that has been learned as a predefined deep learning model; and
Subtype classification that classifies and outputs subtypes corresponding to the causative mechanism of acute ischemic stroke based on the 3D volume data and the segmented lesion area using a subtype classification model that has been learned as a predefined deep learning model including;
A device for diagnosing acute ischemic stroke using a 3D-CNN-based deep learning model.

Images