WO2023243775A1

WO2023243775A1 - Deep learning-based brain lesion detection system using slice image segmentation

Info

Publication number: WO2023243775A1
Application number: PCT/KR2022/014959
Authority: WO
Inventors: 김성헌; 김윤; 김철호; 장재원; 이승아; 최현수
Original assignee: 주식회사 지오비전
Priority date: 2022-06-14
Filing date: 2022-10-05
Publication date: 2023-12-21
Also published as: KR20230171720A

Abstract

The present invention relates to a brain lesion detection system characterized in that a brain lesion area is detected from a volumetric image of a brain using a deep learning module, wherein the volumetric image is divided into N slice images (where N is a natural number equal to or greater than 2) and used as input images for the deep learning module.

Description

Deep learning-based brain lesion detection system using slice image segmentation

The present invention relates to a deep learning-based brain lesion detection system using slice image segmentation.

Acute Ischemic Stroke (AIS) refers to necrosis of brain tissue (cerebral infarction) caused by insufficient supply of blood and oxygen to the brain due to blocked arteries.

Such acute ischemic stroke is a major cause of disability, and it is very important to quickly and accurately determine treatment strategies to prevent disability.

Extending the time for Thrombolysis in Emergency Neurological Deficits (EXTEND), thrombectomy in strokes within 6 to 16 hours of onset selected by perfusion imaging (Endovascular therapy following imaging evaluation for ischemic stroke 3, DEFUSE 3), between neurological deficits and cerebral infarction In recent clinical trials, such as thrombectomy (Diffusion Weighted Image or CT assessment with clinical mismatch in the triage of wake-up and late presenting strokes undergoing neurointervention with Trevo, DAWN) in strokes occurring within 6 to 24 hours of onset showing a discrepancy in The importance of distinguishing lesions from normal brain tissue from brain images and measuring lesion volumes was shown. Diffusion-weighted imaging (DWI) is used to determine where an infarction has occurred in the brain.

Diffusion-weighted imaging is a type of magnetic resonance imaging (MRI) based on measuring the random Brownian motion of water molecules within tissue voxels.

In diffusion-weighted imaging, tissues with cellular expansion exhibit low diffusion coefficients, which can be used to detect brain tissue where infarction has occurred or will occur.

Accurately and quickly segmenting the infarcted lesion in diffusion-weighted images and confirming the volume of the infarcted lesion to determine the amount of drug to be injected are the basis for complete treatment of the patient, and accordingly, diffusion-weighted images Various methods have been developed to automatically segment and detect brain lesion areas.

However, conventionally proposed methods have many problems such as too many tasks that require manual handling by humans, various subtypes of brain lesions, generation of artifacts due to movement, multifocal distribution, and unclear boundaries with normal tissue. .

Therefore, a method is needed to objectively and quickly detect brain lesions in diffusion-weighted images without human intervention.

One object of the present invention is to provide a brain lesion detection system that can detect brain lesion occurrence area and volume from diffusion-weighted images of the brain using a deep learning module.

Meanwhile, other unspecified purposes of the present invention will be additionally considered within the scope that can be easily inferred from the following detailed description and its effects.

To achieve the purpose proposed above, we propose the following solutions.

The brain lesion detection system according to an embodiment of the present invention detects the brain lesion area from a volumetric image of the brain using a deep learning module, and converts the volumetric image into N slice images (where N is a natural number of 2 or more). It is characterized by dividing it and using it as an input image for the deep learning module.

In one embodiment, the volumetric image of the brain may be a diffusion weighted image.

In one embodiment, the deep learning module includes an encoder and a decoder that are symmetrical to each other and have a skip connection, the encoder determines the context by downsampling the spatial dimension, and the decoder upsamples the encoder output image. Increases the spatial dimension by performing, wherein the encoder includes 2D convolution, Relu (Rectified Linear Unit), batch normalization, 2 2 It may be characterized as including upward convolution.

In one embodiment, it further includes a volume estimation module for estimating the volume of the brain lesion area detected by the deep learning module, wherein the volume estimation module calculates accumulation coefficients for the width and height of the original in the slice image. Using the thickness of the slice image, determine the original volume of one pixel in the slice image, count the number of pixels in the brain lesion area in all slices, and calculate the original volume of one pixel in the slice image to the number of pixels in the brain lesion area. It may be characterized by estimating the volume of the brain lesion area by multiplying the volume.

Acute ischemic strokes are relatively small compared to the size of the brain and are rarely located. Therefore, when detecting brain lesions using a conventional deep learning module to confirm acute ischemic stroke, problems of class imbalance and data shortage occur.

The brain lesion detection system according to an embodiment of the present invention divides the diffusion-weighted image of the brain, which is a volumetric image, into slice images and uses them as input images to increase the amount of data and at the same time view overall brain cross-sectional information. It can solve class imbalance and data shortage problems.

In addition, the brain lesion detection system according to an embodiment of the present invention has excellent detection performance and can estimate the volume of the brain lesion using pixels of the predicted mask.

Meanwhile, it is to be added that even if the effects are not explicitly mentioned herein, the effects described in the following specification and their potential effects expected from the technical features of the present invention are treated as if described in the specification of the present invention.

Figure 1 is a diffusion weighted image of the brain, illustrating (a) a case where the brain lesion area is large and (b) a case where the brain lesion area is small.

Figure 2 is a schematic flow chart of a method for detecting a brain lesion area and estimating its volume using a brain lesion detection system according to an embodiment of the present invention.

Figure 3 is an example of learning data augmentation for training the deep learning module of the brain lesion detection system according to an embodiment of the present invention.

Figure 4 is a reference diagram for explaining a method of estimating the volume of a brain lesion area detected using a brain lesion detection system according to an embodiment of the present invention.

Figure 5 is an example image of a small AIS, showing (a) internal verification and (b) external verification.

Figure 6 shows (a) expert annotation, (b) detection result of the example, and (c) detection result of the comparative example regarding the brain lesion area, including (1) large AIS, (2) small AIS, and (3) rarely located small AIS. This is the visual result. (TP is red, FP is green, FN is yellow)

The attached drawings are intended as reference for understanding the technical idea of the present invention, and are not intended to limit the scope of the present invention.

Hereinafter, with reference to the drawings, we will look at the configuration of the present invention guided by various embodiments of the present invention and the effects resulting from the configuration. In describing the present invention, if it is determined that related known functions may unnecessarily obscure the gist of the present invention as they are obvious to those skilled in the art, the detailed description thereof will be omitted.

The term “module” used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions.

In this document, “modules” or “nodes” use computing devices such as CPUs, APs, etc. to perform tasks such as moving, storing, and converting data. For example, a “module” or “node” can be implemented as a device such as a server, PC, tablet PC, smartphone, etc.

The present invention relates to a brain lesion detection system that can detect a brain lesion area from a volumetric image of the brain using a deep learning module and simultaneously estimate its volume. Meanwhile, an example of a volumetric image of the brain may be a concentration-weighted image, but the present invention is not limited thereto.

For the treatment of acute ischemic stroke, it is very important to estimate the volume of the brain lesion area from volumetric images of the brain. Since the brain lesion area is relatively small and sparsely located compared to the size of the brain, a method of detecting the brain lesion area and estimating its volume in a deep learning module using volumetric images taken of the brain directly as input data (hereinafter referred to as “direct When using the "volume estimation method"), when extracting a small patch, there is a problem that the patch does not include the brain lesion area, resulting in serious class imbalance.

Accordingly, the brain lesion detection system of the present invention changes the volumetric image taken of the brain into a slice image instead of the direct volume estimation method, and from there, the deep learning module detects the brain lesion area and estimates the volume from it (hereinafter referred to as the “indirect volume estimation method”) "), class imbalance and data shortage were resolved, resulting in high performance and increased accuracy of volume estimation.

Below, we will compare the direct volume estimation method (comparative example) and the indirect volume estimation method (example).

Diffusion-weighted imaging (DWI) for acute ischemic stroke (AIS) was acquired at Hallym University Chuncheon Sacred Heart Hospital (HUCSH) and Kangwon National University Hospital (KNUH). For HUCSHH, diffusion weighted imaging (DWI) for acute ischemic stroke (AIS) acquired from January 2011 to December 2019, and for KNUH, diffusion weighted imaging (DWI) for acute ischemic stroke (AIS) acquired from July 2014 to October 2019. Diffusion weighted imaging (DWI) for stroke (AIS). Finally, 2,159 DWI images of AIS (69.8 ± 12.7 years for males for HUCSHH; 555 males and 425 females for KNUH; mean ± standard deviation age 72.4 ± 12.4 years) were obtained. Additionally, 121 DWI images from healthy participants as controls (for HUCSHH, 50 men and 52 women with a mean ± SD age of 59.2 ± 16.4 years; for KNUH, 10 men and women with a mean ± SD age of 44.0 ± 17.1 years) 11 people) were obtained. DWI may show no lesions in healthy participants or a b-value of 1,000 s/mm ² while some old lesions may be present if a b-value of 0 s/mm ² . This study was approved by the institutional review boards of HUCSHH and KNUH, which waived the requirement for informed consent (approval numbers HUCSHH 2021-06-013 and KNUH-A-2021-021-001).

MRI to acquire diffusion weighted imaging (DWI) was performed using a variety of machines, including 1.5T (Siemens Helsineers, Erlangen, Germany) and 3T scanners (Ingenia CX, Philips Healthcare, Akechiba, Philips Healthcare, Best, Netherlands). was carried out. The parameters of the DWI sequence are as follows.

DWI sequence parameters

repetition time, 3,000 ms-8,000 ms; echo time, 56 ms-103 ms; flip angle, 90°; matrix, 256 x 256-512 x 512; Field Of View, 220 x 220-256 x 256 mm; number of excitations, 1-5; number of slices, 20-50; slice thickness, 3 mm; b-value, 1,000 s/mm ²

All lesions seen on DWI were manually segmented using the open source software application ITK-Snap by one neurologist (C.K) from HUCSHH and two neurologists (S.H.K and J.-W.J) from KNUH. It has been done. All images with masks were saved in DICOM (Digital Imaging and Communications in Medicine) format (see Figure 1). Figure 2 shows a brain lesion area detected using a brain lesion detection system according to an embodiment of the present invention; This is a schematic flow chart of how to estimate the volume.

The brain lesion detection system of the present invention first converts a volumetric image (eg, DWI) of the brain into a slice image. A volumetric image of the brain is cut into sections at set intervals (or thicknesses) to create N slice images (where N is a natural number of 2 or more).

The brain lesion detection system (indirect volume estimation method) of the present invention uses the generated slice image as an input image to the deep learning module.

Meanwhile, when training a deep learning module, a large amount of data is required. However, since brain lesions are not common diseases, there is a limit to the amount of data. The brain lesion detection system of the present invention performs data augmentation as shown in FIG. 3 to solve the problem of insufficient data. Specifically, random horizontal and vertical flipping and random 90-degree rotation were performed on the slice images. Furthermore, the image size was reduced (e.g., 256

After slicing the volumetric image of the brain and performing augmentation and image size reduction steps, a step of detecting the brain lesion area by segmenting the brain lesion area using a deep learning module is performed.

The deep learning module used in the present invention is based on 3D U-Net, a CNN (Convolutional Neural Networks)-based architecture, and is characterized by using 2D convolution instead of 3D convolution.

Specifically, the deep learning module used in the present invention includes an encoder and a decoder that are symmetrical to each other and have a skip connection, the encoder determines the context by downsampling the spatial dimension, and the decoder performs upsampling on the encoder output image. Increasing the spatial dimension, the encoder includes 2D convolution, Relu (Rectified Linear Unit), batch normalization, and 2 x 2 max pooling layers, and the decoder includes 2D convolution, Relu, batch normalization, and 2 x 2 up convolution. Includes. Convolution, Relu, batch normalization, etc. can be used as known methods, and detailed explanations are omitted here.

The deep learning module of the present invention changes the volumetric image of the brain into a slice image and inputs it, thereby increasing the amount of data and making it possible to view overall brain cross-sectional information, thereby solving problems caused by hierarchical imbalance and lack of data.

Additionally, the deep learning module of the present invention uses two adjacent slice images as three-channel input. When converting a volumetric image of the brain into a slice image and inputting the slice image through one input channel, there is a problem that the entire context information of the volumetric image is missing. The deep learning module of the present invention has the effect of providing additional information such as context information and volume information by inputting two adjacent slice images in three channels.

The volume of the brain lesion area is estimated using the result of detecting the area separated as the brain lesion area from the slice image input from the deep learning module.

Estimating the volume of the detected brain lesion area is performed by the volume estimation module.

As mentioned earlier, the slice image input to the deep learning module can be resized for performance. Therefore, in order to estimate the volume of the lesion area, the volume estimation module determines the scale coefficients sf _w and sf _h for the width and height of the original slice image, and the spacing (or thickness) of the slice image in the header file of the image file (DICOM header file). ) to obtain the interval information si. The volume estimation module calculates the volume (V) of the pixel through each acquired accumulation coefficient and the interval of the slice image.

Next, the volume estimation module calculates the total number of pixels (P) corresponding to the brain lesion area by accumulating the number of pixels (p) corresponding to the brain lesion area in each slice image output from the deep learning module. H, W, and N are the height, width, and number of slide images, respectively.

Finally, the volume estimation module estimates the total volume of the brain lesion area by multiplying the total number of pixels (P) corresponding to the brain lesion area by the pixel volume (V).

Meanwhile, the loss function of the deep learning module of the present invention is learned using a dice loss loss function, and is learned based on knowledge based on diffusion weighted imaging (DWI) of acute ischemic stroke.

Here, y _true is the label for the pixel value, and y _pred is the prediction probability for the pixel value.

Meanwhile, in the direct volume estimation method of the comparative example, 3D U-Net, a CNN (Convolutional Neural Networks)-based architecture for deep learning modules, was used. 3D U-Net has a U-shaped structure and uses symmetric encoders and decoders with skipped connections. The encoder determines the context by downsampling the spatial dimension. The encoder consists of three layers: two 3D convolutions, a Rectified Linear Unit (Relu) and batch normalization, and a 2 x 2 max pooling layer. The decoder performs upsampling on the encoder output image to increase the spatial dimension. The decoder consists of three layers, including two 3D convolutions, Relu and batch normalization, and a 2 x 2 up convolution. Symmetric blocks are used with skip concatenation of the corresponding encoding blocks to make up for lost context. Skip connection adds a layer input to the output to compensate for information lost during the convolution process, which helps improve data recovery performance and perform semantic segmentation well. Training of 3D segmentation models requires high computation time and memory, so patch-based 3D segmentation was performed to alleviate this problem.

To compare the performance of the example (indirect volume estimation method) and the comparative example (direct volume estimation method), precision (see Equation 4), recall rate (see Equation 5), F1-score (see Equation 6), and Jaccard index ( Equation 7) was adopted.

Since the important thing in a brain lesion detection system is to accurately detect the brain lesion area, here we mainly set the metric for true-positive (TP).

Here, TP is the number of pixels in the result correctly predicted as the brain lesion area (AIS), false-positive (FP) is the number of pixels in the result incorrectly predicted as the brain lesion area (AIS), and false negative (FN) is the number of pixels in the result incorrectly predicted as the brain lesion area (AIS). False-negative) is the number of pixels that were incorrectly predicted to be not the brain lesion area (AIS).

In addition, volume similarity (VS) (see Equation 8) and mean absolute error (MAE) (see Equation 9) were adopted to evaluate the volume estimation performance of the brain lesion area.

VS estimated the volume of the predicted brain lesion area (AIS) to indicate its similarity to the labeled brain lesion area (AIS). MAE is the average absolute value of the error, which is the difference between the actual value and the predicted value. That is, the error between the predicted brain lesion area (AIS) and the labeled brain lesion area (AIS) is estimated. In Equations 8 and 9, measurement is the area of the predicted brain lesion area (AIS) (TP), and groundtruth is the area of the labeled brain lesion area (AIS).

Both examples and comparative examples were implemented using the Pytorch framework 1.10 in Python 3.8.10 on Ubuntu using four NVIDIA RTX 3090s and were included as the core algorithm of an AI-based medical solution (ZioMed; ZIOVISION, Chuncheon, Korea). Internal validation was performed using data obtained from HUCSHH, and the dataset was randomly divided into training, validation, and test sets in a ratio of 8:1:1. Data obtained from KNUH were used for external validation. Since it takes a lot of time to load the images and interval information of the data set, considering that it takes a long time to load the DICOM file, the images and interval information extracted from the DICOM file were saved in HDF5.

For Examples and Comparative Examples, an experiment was performed to evaluate the segmentation performance of diffusion weighted imaging (DWI) of acute ischemic stroke (AIS). Tables 2 and 3 show the performance results of examples and comparative examples performed for internal and external verification of diffusion weighted imaging (DWI) of acute ischemic stroke (AIS).

Regarding the internal validation data of acute ischemic stroke patients, the F1-score of the example (indirect) using the pre-trained brain glioma model was 76.02%. This was significantly higher than the F1-score of 55.71% and 52.48% in the comparative example (direct) using pre-trained models for automatic implants and tumors, respectively. Meanwhile, the F1-score of the scratch mode example (indirect) is 73.09%, which is higher than the F1-score of 54.76% of the scratch mode comparative example (direct). Among the comparative examples (direct), the F1-score is 73.09%. and showed superior performance than the F1-score of a pre-trained model for tumors.

Regarding the external validation data of acute ischemic stroke patients, the F1-score of the example (indirect) using the pre-trained brain glioma model was 77.23%, and like the internal validation data, it showed superior performance compared to the comparative example. It was. This is also the same for the scratch mode embodiment (indirect).

Likewise, the Jaccard index of the internal and external validation of the example (indirect) using the pre-trained indirect model was 62.12% and 63.82%, respectively, which was higher than that of other models.

Through VS and MAE, it was confirmed whether the results of estimating brain lesion area (AIS) volume in Examples and Comparative Examples were reliable. Table 4 shows the results of estimating the volume of the brain lesion area (AIS) detected for internal and external verification. The VS and MAE, the volume estimation results of the direct example using the pretrained cranial glioma model, were 67.68% and 1.159 cc, respectively, in internal validation, and 62.59% and 5.706 cc, respectively, in external validation. In comparison, the VS and MAE, which are the volume estimation results of the example (indirect), were 93.25% and 0.797cc, respectively, in internal verification, and 89.17% and 2.468cc, respectively, in external verification, which was significantly higher than that of the comparative example.

Additionally, the best mode embodiment was validated against a normal group with no AIS at all to check for FP errors. As a result, in the case of internal and external verification, the MAE values were 0.028cc and 0.009cc, respectively, showing very small errors.

Meanwhile, these results show that when learning the deep learning module of the example from diffusion weighted imaging (DWI) of acute ischemic stroke (AIS), it has the best performance and accuracy when pre-trained using data on brain glioma. Able to know.

Figure 5 is an image example of a small AIS, showing examples of (a) internal verification and (b) external verification, and Figure 6 shows (a) expert annotation regarding the brain lesion area, (b) detection results of the example, and (c) ) As the detection results of the comparative example, the visual results are for (1) large AIS, (2) small AIS, and (3) rarely located small AIS (red for TP, green for FP, yellow for FN). As seen above, even with reference to FIGS. 5 and 6, it can be seen that the embodiment has higher accuracy than the comparative example.

In the comparative example, the volume of the brain lesion area is derived from a volumetric image (3D) of the brain without additional processing, so the comparative example is faster than the example using a slice image (2D) cut into cross-sections of the volumetric image of the brain. expected that more accurate volume estimation would be possible even if it was slower, but this was not the case. In other words, the Example had higher performance in detecting the brain lesion area than the Comparative Example, and at the same time, the accuracy of estimating the volume of the brain lesion area was also high.

The brain lesion detection system as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided on a non-transitory computer readable medium. Here, a non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above may be stored and provided in non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

The scope of protection of the present invention is not limited to the description and expression of the embodiments explicitly described above. In addition, it is once again added that the scope of protection of the present invention may not be limited due to changes or substitutions that are obvious in the technical field to which the present invention pertains.

Claims

The brain lesion area is detected from a volumetric image of the brain using a deep learning module, and the volumetric image is divided into N slice images (where N is a natural number of 2 or more) and used as the input image of the deep learning module. Brain lesion detection system.
According to paragraph 1,

A brain lesion detection system, wherein the volumetric image of the brain is a diffusion weighted image.
According to paragraph 1,

The deep learning module includes an encoder and a decoder that are symmetrical to each other and have a skip connection, the encoder determines the context by downsampling the spatial dimension, and the decoder performs upsampling on the encoder output image to determine the spatial dimension. Increasing, the encoder includes 2D convolution, Relu (Rectified Linear Unit), batch normalization, 2 x 2 max pooling layer, and the decoder includes 2D convolution, Relu, batch normalization, 2 x 2 up convolution. A brain lesion detection system characterized by:
According to paragraph 1,

Further comprising a volume estimation module for estimating the volume of the brain lesion area detected by the deep learning module,

The volume estimation module determines the original volume of one pixel in the slice image using the accumulation coefficient for the width and height of the original in the slice image and the thickness of the slice image, and determines the original volume of one pixel in the slice image, and the number of pixels in the brain lesion area in all slices. A brain lesion detection system characterized by counting and estimating the volume of the brain lesion area by multiplying the number of pixels in the brain lesion area by the original volume of one pixel in the slice image.