WO2020263002A1

WO2020263002A1 - Blood vessel segmentation method

Info

Publication number: WO2020263002A1
Application number: PCT/KR2020/008319
Authority: WO
Inventors: 조한용; 권순성
Original assignee: 에이아이메딕 주식회사
Priority date: 2019-06-27
Filing date: 2020-06-26
Publication date: 2020-12-30
Also published as: KR102267219B1; KR20210001233A

Abstract

The present invention relates to a blood vessel segmentation method. More specifically, the present invention relates to a method for segmenting a blood vessel region by processing a plurality of two-dimensional blood vessel tomography images by a deep learning method. A blood vessel region segmentation method according to the present invention is provided. The segmentation method according to the present invention comprises the steps of: receiving, as an input, a plurality of two-dimensional tomography images; preprocessing the plurality of two-dimensional tomography images received as the input to mark a region in which a blood vessel is located to thereby generate training image data; performing learning with the generated training image data to generate a blood vessel feature image prediction model; and inputting the plurality of two-dimensional tomography images into the generated blood vessel feature image prediction model to receive, as an output, a plurality of two-dimensional tomography images displaying blood vessel features.

Description

Blood vessel segmentation method

The present invention relates to a vascular segmentation method. More specifically, the present disclosure relates to a method of segmenting a blood vessel region by processing a plurality of 2D blood vessel tomography images by a deep running method.

A medical image processing device is a device that acquires an image capable of non-invasively showing the internal structure of a human body. The medical image output from the medical image processing device may be analyzed and used to diagnose a patient's disease.

Devices for photographing and processing medical images include Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Single Photon Emission Computed Tomography (SPECT), and positron tomography. PET, Positron Emission Tomography) and Ultrasound.

CT and MRI are widely used for the diagnosis of cerebrovascular diseases. Since the causes of cerebrovascular diseases are diverse and treatment methods and prognosis may vary depending on the patient, the exact cause analysis, determination of appropriate treatment methods, and prognosis Various imaging techniques are being developed for prediction.

The method using CT has the disadvantage of not knowing the extent of the cerebral infarction accurately and the use of radiation exposure and contrast media, and MRI can determine the extent of the cerebral infarction more accurately, but it takes a relatively long time to obtain an image and requires emergency such as acute cerebral infarction. It can be limited in the situation, is very sensitive to the patient's movements, and is relatively safer than CT, but it also has the disadvantage of requiring a contrast medium.

In order to diagnose and analyze vascular lesions such as cerebral infarction or cerebral hemorrhage or coronary artery stenosis, segmentation to generate a three-dimensional shape model of a blood vessel by processing a plurality of two-dimensional tomographic images is required. In particular, for accurate and rapid diagnosis, a method of accurately and quickly segmenting a blood vessel region is required.

Recently, a medical image processing technology using a deep running or machine learning technique has been developed. In particular, development to diagnose diseases by applying deep running techniques to medical images acquired from devices such as X-ray, ultrasound, CT (Computed Tomography), MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography), etc. It is going on. That is, an auxiliary diagnostic system has been developed to classify whether the tissue shown in medical images is normal or abnormal, and in the case of tumors, whether it is positive or negative using deep running techniques, and it is known that it has been developed to the level in which radiology doctors can read images. .

Algorithms such as naive bayes, SVM (Support Vector Machine), ANN (Artificial Neural Network), and HMM (Hidden Markov Model) are known as algorithms for automatically classifying the presence or absence of such lesions. In addition, machine learning algorithms can be used for this classification, and machine learning algorithms are largely classified into supervised learning and unsupervised learning algorithms.

There is a need for a technology to accurately and quickly segment a blood vessel region by processing medical images using deep running or machine learning algorithms. An object of the present invention is to provide a method for segmenting a blood vessel region by processing a plurality of tomography medical images.

The present invention provides a method for segmenting a blood vessel region. The segmentation method according to the present invention includes the steps of receiving a plurality of 2D tomography images, preprocessing the received 2D tomography images to display an area where a blood vessel is located to generate training image data, and Generating a blood vessel feature image prediction model by learning from the training image data, and inputting a plurality of two-dimensional tomographic images into the generated blood vessel feature image prediction model to output a plurality of two-dimensional tomographic images displaying blood vessel features. do.

It is preferable to use a U-net algorithm to learn the blood vessel feature image prediction model.

In addition, it is more preferable to use the GAN algorithm in the learning of the blood vessel feature image prediction model, and to use the U-net algorithm as a generator module in the GAN algorithm.

In addition, in the step of learning the blood vessel feature image prediction model, the GAN algorithm is used, and the U-net algorithm is used as a generator module in the GAN algorithm, first initial learning of U-net, and initial learning of the U-net. Upon completion, it is more preferable to consider the MR segmentation image output from U-net as a fake image, and to learn simultaneously with the inspector module.

According to the present invention, there is provided a method of segmenting a blood vessel region by processing a plurality of two-dimensional tomographic images to accurately and quickly diagnose and analyze a lesion of a blood vessel such as cardiovascular or cerebrovascular disease.

1 is a schematic diagram of a conventional blood vessel modeling method

2 is an MRI image showing (a) an original image, (b) an image for learning, (c) an original image preprocessing result, and (d) a training image preprocessing result.

3 is a schematic diagram of a U-net learning algorithm according to the present invention

4 is a predicted cerebrovascular output image using a prediction model

5 is a schematic diagram of a cerebrovascular target method using the windowing technique of the present invention

6 is a schematic diagram of a U-net architecture according to the present invention

7 is a schematic diagram of a GAN algorithm according to the present invention

8 is a change diagram of probability distribution when applying the GAN algorithm according to the present invention

9 is a schematic diagram of an embodiment of a GAN algorithm according to the present invention

In the present specification, "image" may mean multi-dimensional data composed of discrete image elements (eg, pixels in a 2D image and voxels in a 3D image). For example, the image may include a medical image of an object acquired by an MRI or CT imaging apparatus.

In the present specification, the "object" may include a human or an animal, or a part of a human or animal. For example, the subject may include organs such as liver, heart, uterus, brain, breast, and abdomen, or blood vessels. Further, the "object" may include a phantom. The phantom refers to a material having a volume very close to the density and effective atomic number of an organism, and may include a sphere-shaped phantom having properties similar to the body.

In the present specification, the "user" may be a medical expert, such as a doctor, a nurse, a clinical pathologist, a medical imaging expert, and the like, and may be a technician who repairs a medical device, but is not limited thereto. The CT image may be a cardiovascular or cerebrovascular image, but is not limited thereto, and any tomography image including a blood vessel may be used. Although the brain MRA image was used as an example in the detailed description of the present invention, it should be understood as an exemplary.

1 shows a conventional method for modeling cerebrovascular 3D. First, a commercial medical image viewer receives MRA data (DICOM file) and outputs two to three hundred two-dimensional images in an axial view. Next, by adjusting the intensity threshold of the output axial view image, cerebrovascular and tissues similar to cerebrovascular intensity are primarily divided. Next, stenosis is implemented, disconnected blood vessels are connected, and tissues other than blood vessels are removed according to manual work in the divided cerebrovascular shape. In this case, the worker needs segmentation know-how and anatomical knowledge. Next, the mesh is created to complete the grid for computer simulation.

A method of segmenting a blood vessel region of a blood vessel according to the present invention is a method of segmenting a plurality of 2D tomographic images using a deep running technique.

2 shows an image for learning according to the segmentation method according to the present invention. Fig. 2(a) is an original image, Fig. 2(b) is an image showing a blood vessel area for learning (Ground Truth), Fig. 2(c) is an image preprocessed of the original image, and Fig. 2(d) is a training image This is the pre-processed MRI image.

4 shows a result of automatically extracting a cerebrovascular region from an MR image predicted by an artificial intelligence system according to an embodiment of the present invention, and the green part represents the cerebrovascular region.

Hereinafter, the segmentation method according to the present invention will be described in detail.

The segmentation method according to the present invention includes the steps of receiving a plurality of 2D tomography images, preprocessing the received 2D tomography images to display an area where a blood vessel is located to generate training image data, and Generating a blood vessel feature image prediction model by learning from the training image data, and inputting a plurality of two-dimensional tomographic images into the generated blood vessel feature image prediction model to output a plurality of two-dimensional tomographic images displaying blood vessel features. .

First, a model capable of obtaining a 2D tomography image showing a blood vessel region is trained by performing machine learning using a 2D tomography image showing a blood vessel region. The FCN (Fully Convolutional Network) algorithm is used for model training. The FCN model uses a method of properly mixing by upsampling the values of the convolutional lower pooling layer so that the output is not a class value but a pixel heat map. Based on the FCN model, an algorithm that approaches the global optimal function is implemented by learning using pairs of data consisting of an input (raw CT image) and a result (clinically verified cerebrovascular segmentation image).

<U-net algorithm>

U-net algorithm can also be used as a machine learning algorithm. The U-net algorithm is an algorithm developed based on FCN, and has the characteristic of obtaining more accurate segmentation results even with little data. As shown in FIG. 3, U-net is a name given because it has a U-shaped shape, and the left is called a contracting path and the right is called an expansive path based on the center of the network. In FIG. 3, a blue box indicates a multi-channel feature map, and each arrow indicates a different operation for each color. The red arrow is max pooling, the yellow arrow is up-convolution, and the green is copy and crop, which is a concept of skip connection.

The reason for using skip connection is that the deeper the layer is, the more information on the local feature is lost and the information on the global feature becomes dominant due to the nature of deep running.As a result, the location of the cerebrovascular vessels from the MRA image is well located, but the area of the blood vessels is detailed. This is because the possibility of not being able to extract accurately increases. In the structure of U-net, the contracting path helps to capture the context of the image, and the expansive path up-sampling the feature map and combining it with the context of the feature map captured in the contracting path performs more accurate localization. In the present invention, this characteristic is the main idea of U-net, and an important point different from the existing FCN algorithm is that the number of feature channels is larger in the up-sampling process.

4 is an image showing a cerebrovascular area generated by applying the U-net algorithm. Since cerebrovascular blood vessels are a three-dimensional structure that connects from the top to the bottom of the head, rather than determining the location of cerebrovascular blood vessels in a single MR image and making them regions, information on the distribution of cerebrovascular vessels by considering all the front and rear of the image to be region. Was applied to learning. It is assumed that 150 MR images included in one case of cerebrovascular blood vessels used for learning are a collection of images showing a cross section moving from the top to the bottom of the head. Therefore, instead of a single MR image, a windowing technique was used in which multiple MR images were bundled and context information was input to the development network to learn data without a repetitive structure. A stack was constructed by stacking a single MR image as a target, and k images in the +Z-axis direction and k-images in the -Z-axis direction of the image. When the size of the MR image in the X-axis and Y-axis directions is defined as x and y, this stack is composed of (x, y, 2k+1) dimensional tensors. Therefore, the window stack is input to the U-net algorithm, not a single MR image, and the channel size of the neural network input unit is 2k+1. Specifically, the cerebrovascular vessel is a structure that is connected from the top to the bottom without a disconnected area, and when comparing through continuous images constituting a window, overlapping sections are necessarily generated. When the distributions of these sections are overlapped, only the cerebrovascular area, which is the target of regionalization, appears prominently as shown in the stacked-distribution shown in FIG. 5.

The contracting path of the U-net according to the present invention follows a general convolutional network, and performs two repeated 3x3 convolution operations including stride 2, 2x2 max pooling operation and Relu function for down-sampling. In summary, it is calculated in the order of 3x3convolution-Relu-2x2max pooling-3x3convolution-Relu-2x2max pooling, and the number of feature map channels doubles during the down-sampling process. This is concatenated with the feature map created in the expansive path and the feature map created in the contracting path.

The U-net model according to the present invention uses ADAM (Adaptive Moment Estimation) among gradient descent algorithms to learn to find a variable that minimizes cross entropy. Cross entropy is a value that expresses two different probability distributions describing the same event, and is a measure of how close the probability distribution of the model is to the probability distribution of the actual label in the artificial intelligence learning process.

The system developed according to the present invention, in one embodiment, processed a total of 70 case data to proceed with learning of the artificial intelligence system. Of these, 50 case MR images were amplified into 360 cases and used for learning, and 15 cases were used as verification data. In addition, in order to learn U-net, a total of 100,000 iterations were performed, and various variables were tested to obtain parameter values optimized for cerebrovascular regionalization. In deep running, the weight (weight) and bias (bias) values were learned by themselves to construct an optimal neural network, and the following variables (hyperparameters) were tuned.

-Learning rate: The learning rate is a variable that determines how much error of the result is reflected in learning. If the learning rate is high, the result is likely to vibrate without convergence. If the learning rate is low, the learning rate is slow and there is a possibility that convergence to the local minimum.

-Cost function: The cost function is a function that calculates the difference between the expected value and the actual value according to the input. The developer decides which function to use among various cost functions so that the problem to be solved by artificial intelligence can be efficiently learned. Typical examples include mean square error and cross entropy error.

-Mini-batch size: It takes a lot of time to calculate the cost function of all data, so some of the data are used to update the weights. At this time, if the size of the mini-batch is large, the learning speed can be increased, and if the size of the mini-batch is small, the weight can be updated more frequently, so the accuracy of the neural network may vary.

-Training repetition number: If the number of training is too many, the actual accuracy of the test data may decrease even if the accuracy is increased during training due to overfitting.

-Number of hidden unints: If the number of units of the hidden layer is large, the expressive power of the network becomes wider, which may result in better performance, but there is a problem that may result in overfitting. Conversely, if it is too small, it may be underfitting.

-Dropout: Dropout omits part of the network. Randomly omitting neurons in some networks, reducing the likelihood of overfitting. There is a difference in neural network performance depending on the proportion of neurons that are omitted.

In addition to this, regularization parameters and weight initialization methods are set through trial and error.

In addition, it is more preferable to use the GAN algorithm in the learning of the blood vessel feature image prediction model, and to use the U-net algorithm as a generator module in the GAN algorithm. In addition, in the step of learning the blood vessel feature image prediction model, the GAN algorithm is used, and the U-net algorithm is used as a generator module in the GAN algorithm, first initial learning of U-net, and initial learning of the U-net. Upon completion, it is more preferable to consider the MR segmentation image output from U-net as a fake image, and to learn simultaneously with the inspector module.

The GAN algorithm was used to further improve the performance of U-net.

GAN is a pair of mathematical models composed of a generator and a discriminator. The generator and the inspector oppose each other hostilely and are gradually improving each other's performance. The creator tries to deceive the inspector by falsifying data like a banknote counterfeiter, and the inspector is a method of improving performance through efforts to discriminate the forged data from real data.

The generator module of the GAN model according to the present invention becomes a U-Net, and the inspector module is configured to estimate a probability value whether the received data is an output of U-Net or actual data. The relationship between the two modules constituting the GAN algorithm is expressed by the following equation.

After inputting the actual data x into the function D of the inspector module, D(x) is trained so that it becomes 1, and the random noise probability distribution z is converted to the resultant G(z) of the inspector module. Enter the function D and learn so that the value D(G(z)) becomes 0. In this process, the generator module attempts to increase the probability that the inspector module makes a mistake, while the inspector module attempts to reduce the performance of the generator module by identifying the counterfeit product generated by the generator module as false. As the competition between the two modules continues, the creator's ability to forge data and the inspector's ability to discriminate data gradually improve with each other, thereby enhancing the creator's ability one step further.

In FIG. 8, the blue dotted line represents the probability distribution of the inspector, the black dotted line represents the actual data distribution, and the green solid line represents the fake data distribution generated from the generator. As shown in FIG. 8, as the learning progresses, the generator performance gradually improves, and when the green line becomes indistinguishable from the black dotted line, the blue dotted line, which is the probability distribution of the inspector, becomes horizontal, so that real and fake data cannot be distinguished. At this point, the training of the model is terminated, and counterfeit data similar to the real data is generated using only the generator.

The general GAN model trains two network models, the generator module and the inspector module at the same time, and modifies the weight and bias variables that minimize cross entropy by applying the inspector's gradient to the generator. However, the GAN algorithm according to the present invention proceeds to learn by separating into two steps.

9 is a schematic diagram of an embodiment of the GAN algorithm according to the present invention. First, the U-net is initially trained and the output MR segmentation image is regarded as a fake image, that is, U-net is applied as a GAN generator module to proceed with learning. In the initial learning stage, the generator performs a function to find the location of cerebrovascular vessels by learning independently without receiving the examiner's gradient. After the initial learning is finished, learning proceeds with the examiner at the same time, where the generator receives the examiner's gradient and modifies the weights and bias variables again. In this process, additional performance enhancements are made to the generator module, which has already converged and cannot be expected to improve performance any more. The inspector module receives the original MR image and the territorialized data as a set, and determines whether the received territorialized image is real data or a counterfeit output from the creator. MRA image and regionized image are compressed by passing through multi-layer convolution and pooling layers, respectively. At this time, a zero-padding technique is applied to each convolution layer to preserve the original size of the image regardless of the kernel size. Therefore, the image received in the Convolutional-Relu-Pooling step is accurately compressed horizontally and vertically into 1/2 size and transmitted to the next layer. The MRA image and the segmented image that passed through the inspector module are converted into a feature map compressed to 1/32 size, and the feature map of the compressed MRA image and the segmented feature map are combined and delivered to a fully connected layer. . In one embodiment, a total of four fully connected layers were used, and all of the activation functions were Relu functions. Through all these networks, it is determined whether it is a real video or a fake video output from the creator.

The GAN model according to the present invention is implemented using the following equation as described above.

The inspector module learns in the direction of widening the gap between D(Xct,Ygt) and D(Xct,S(ct)), and the generator module reduces the gap between Ygt and S(Xct) and finally D(Xct,S(ct)) is learned in the same direction as each other. Above

Is optimized only by cross entropy during U-Net initial learning, so it is highly likely that it is not the minimum point within the derivative. Therefore, after initial learning, the GAN learning process checks the performance change every short learning section,

While staying near the minimum of cross entropy, finely adjust according to the gradient of the above equation. Compared to initial learning through tests, only a very small number of variables are updated.

It is possible to converge, and the change of parameters is not large in the transfer learning section, which is performed relatively short in the examiner module.

As a result, the GAN algorithm stores the weights and biases of the neural network with improved performance, and completes the development of a network that segments the cerebrovascular area when MRA data is input using this.

Claims

A method of segmenting a blood vessel region using a computer system,

Receiving a plurality of 2D tomography images; and

Generating training image data by pre-processing a plurality of input 2D tomography images to display an area where blood vessels are located;

Learning from the generated training image data to generate a blood vessel feature image prediction model,

And receiving a plurality of two-dimensional tomographic images displaying blood vessel features by inputting a plurality of two-dimensional tomographic images into the generated blood vessel feature image prediction model.
The method of claim 1,

The learning of the blood vessel feature image prediction model is a blood vessel region segmentation method using a U-net algorithm.
The method of claim 1,

In the step of learning the blood vessel feature image prediction model, a GAN algorithm is used, and a U-net algorithm is used as a generator module in the GAN algorithm.
The method of claim 1,

In the learning of the blood vessel characteristic image prediction model, a GAN algorithm is used, and a U-net algorithm is used as a generator module in the GAN algorithm,

First, the U-net is initially learned, and when the U-net is completed, the MR segmentation image output from the U-net is regarded as a fake image, and the blood vessel region segmentation method is learned simultaneously with the inspector module.