CN114066908A - Method and system for brain tumor image segmentation - Google Patents
Method and system for brain tumor image segmentation Download PDFInfo
- Publication number
- CN114066908A CN114066908A CN202111178058.1A CN202111178058A CN114066908A CN 114066908 A CN114066908 A CN 114066908A CN 202111178058 A CN202111178058 A CN 202111178058A CN 114066908 A CN114066908 A CN 114066908A
- Authority
- CN
- China
- Prior art keywords
- brain tumor
- features
- scale information
- image
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000003174 Brain Neoplasms Diseases 0.000 title claims abstract description 91
- 238000003709 image segmentation Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000007246 mechanism Effects 0.000 claims abstract description 15
- 238000004590 computer program Methods 0.000 claims description 15
- 238000011176 pooling Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 3
- 230000004931 aggregating effect Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 description 21
- 206010028980 Neoplasm Diseases 0.000 description 20
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 12
- 238000012549 training Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 7
- 238000002595 magnetic resonance imaging Methods 0.000 description 6
- 201000007983 brain glioma Diseases 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000005303 weighing Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000005481 NMR spectroscopy Methods 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000005013 brain tissue Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 201000007455 central nervous system cancer Diseases 0.000 description 1
- 208000025997 central nervous system neoplasm Diseases 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- JXSJBGJIGXNWCI-UHFFFAOYSA-N diethyl 2-[(dimethoxyphosphorothioyl)thio]succinate Chemical compound CCOC(=O)CC(SP(=S)(OC)OC)C(=O)OCC JXSJBGJIGXNWCI-UHFFFAOYSA-N 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 235000019580 granularity Nutrition 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention belongs to the technical field of medical image segmentation, and provides a method and a system for segmenting a brain tumor image. Extracting multi-scale information of a brain tumor image; respectively carrying out coding operation on the brain tumor image based on the multi-scale information so as to extract high semantic features corresponding to the scale information and generate image representation corresponding to the scale information; extracting low semantic features from the image representation of the multi-scale information, and respectively combining with the high semantic features; respectively carrying out weighted recalibration on the combined features according to a space-channel attention mechanism to obtain corresponding attention features; and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
Description
Technical Field
The invention belongs to the technical field of medical image segmentation, and particularly relates to a method and a system for segmenting a brain tumor image.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Brain tumors are abnormal cells that divide and grow abnormally in brain tissue with high morbidity and mortality rates exceeding 3%. Investigations have shown that brain gliomas account for approximately 27% of central nervous system tumors and 81% of malignant tumors. Brain gliomas are highly invasive and have a variety of different histological subregions, with the size, shape, grade and location of the different tumors varying widely. Currently, neuroradiologists still use manual segmentation to diagnose brain tumors. The method is time-consuming and labor-consuming, is easily influenced by personal subjective factors of experts, and has high misjudgment rate and limited precision. Therefore, an automatic and accurate tool for segmenting brain tumors is urgently needed.
Patients are usually clinically diagnosed, treated and prognostically reviewed using Magnetic Resonance Imaging (MRI) of the brain, the multiple sequences of which help physicians to accurately assess tumors and plan treatment. The inherent heterogeneity of brain gliomas manifests itself as highly non-uniform and irregular shapes on multi-modality MRI images. And the MRI images have the problems of poor image resolution, low contrast, artifacts, and noise caused by imaging differences among different imaging devices. For MRI brain tumor images, the conventional segmentation methods mainly use wavelet transform, level set method, watershed method, lawy method, and image atlas method. For example, a level set method combining intensity distribution information and gradient information in a brain glioma image is adopted, so that the distribution position and the prominent edge information of the tumor are considered while the low-dimensional topologic property is ensured. The method can well combine the prior knowledge, but needs manual intervention to correct the result and carry out post-processing on the algorithm. With the development of medical image processing, computer vision and other technologies, the automation of brain tumor segmentation is further improved by a machine learning-based method, and the defects of manual intervention in the traditional method, such as K-means clustering, support vector machine, random forest and other methods, are overcome. However, the inventor finds that the method based on machine learning has the problems of sensitivity to noise, easiness in over-segmentation, obvious over-fitting phenomenon and incapability of balancing noise resistance and precision, so that the segmentation precision of the brain tumor image is influenced.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a method and a system for segmenting a brain tumor image, which fully extract brain tumor features in a brain nuclear magnetic resonance image by combining space-channel attention and residual grouping convolution, and have better segmentation performance by combining the space feature information and channel feature information of an image tumor feature and attention residual grouping convolution neural network.
In order to achieve the purpose, the invention adopts the following technical scheme:
a first aspect of the invention provides a method for brain tumor image segmentation, comprising:
extracting multi-scale information of the brain tumor image;
respectively carrying out coding operation on the brain tumor image based on the multi-scale information so as to extract high semantic features corresponding to the scale information and generate image representation corresponding to the scale information;
extracting low semantic features from the image representation of the multi-scale information, and respectively combining with the high semantic features;
respectively carrying out weighted recalibration on the combined features according to a space-channel attention mechanism to obtain corresponding attention features;
and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
A second aspect of the invention provides a system for brain tumor image segmentation.
In one or more embodiments, a system for brain tumor image segmentation, comprising:
the multi-scale information extraction module is used for extracting multi-scale information of the brain tumor image;
the information coding module is used for respectively carrying out coding operation on the brain tumor images based on the multi-scale information so as to extract high semantic features corresponding to the scale information and generate image representations corresponding to the scale information;
a feature combination module for extracting low semantic features from the image representation of the multi-scale information, respectively combined with the high semantic features;
the attention characteristic determination module is used for respectively carrying out weighted recalibration on the space and channel dimensions of the combined characteristics based on a space-channel attention mechanism to obtain corresponding attention characteristics;
and the image segmentation module is used for restoring the attention characteristics to the original resolution and obtaining a brain tumor image segmentation result through the characteristic representation of the image.
In some other embodiments, a system for brain tumor image segmentation, comprising:
an image acquisition unit for acquiring a brain tumor image;
the image segmentation unit is used for obtaining a brain tumor image segmentation result based on the brain tumor image and a brain tumor image segmentation model trained in advance;
wherein the brain tumor image segmentation model comprises:
a multi-scale input module for extracting multi-scale information of the brain tumor image;
an encoder module for performing encoding operations on brain tumor images, respectively, based on the multi-scale information, to extract high semantic features corresponding to the scale information and generate image representations corresponding to the scale information;
a skip connection module for connecting the high semantic features extracted by the encoder module and the low semantic features extracted by the decoder module from the image representation of the multi-scale information to obtain combined features;
a decoder module, in which a space-channel attention mechanism is embedded, and the combined features are respectively weighted and re-calibrated in space and channel dimensions to obtain corresponding attention features; and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
A third aspect of the invention provides a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for brain tumor image segmentation as set forth above.
A fourth aspect of the invention provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for brain tumor image segmentation as described above when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
according to the method, the characteristics of the brain tumor image can be fully extracted from the multi-scale information of the brain tumor image, and the brain tumor image is respectively subjected to coding operation based on the multi-scale information so as to extract the high semantic characteristics corresponding to the scale information and generate the image representation corresponding to the scale information; extracting low semantic features from the image representation of the multi-scale information, and respectively combining with the high semantic features; and respectively carrying out weighted recalibration on the space and channel dimensions of the combined features based on a space-channel attention mechanism to obtain corresponding attention features, so that the tumor region is concerned more, the channel information and the space information of the target are concerned more, and irrelevant regions are inhibited to improve the segmentation accuracy.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a flow chart of a method for brain tumor image segmentation in accordance with an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a system for brain tumor image segmentation according to a second embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a system for segmenting a brain tumor image according to a third embodiment of the present invention;
FIG. 4 is a flowchart of convolution network training based on a convolution block attention residual grouping according to an embodiment of the present invention;
FIG. 5 is an architecture diagram of a convolution block attention residual based packet convolution network according to an embodiment of the present invention;
FIG. 6(a) is a generic convolution unit;
FIG. 6(b) is a residual convolution unit;
FIG. 6(c) is a residual block convolution unit;
FIG. 7 is a convolution block attention module of an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example one
Referring to fig. 1, the present embodiment provides a method for brain tumor image segmentation, which specifically includes the following steps:
s101: and extracting multi-scale information of the brain tumor image.
Where so-called multiscale is actually a sampling of different granularities of information. Different features are usually observed at different scales, and in networks, high-level images have stronger semantic information, but lose much image information in a continuous convolution process. Therefore, we perform a maximum pooling operation on the input image to obtain feature images of different sizes, which have less convolution and therefore contain more detail information. The image segmentation method carries out addition operation on the image and an image output by a corresponding down-sampling layer, and effectively fuses information of the image and the image to improve the performance of model segmentation.
S102: and respectively carrying out coding operation on the brain tumor image based on the multi-scale information so as to extract high semantic features corresponding to the scale information and generate image representation corresponding to the scale information.
Wherein, the brain tumor image is encoded by an encoder.
And extracting high semantic features of corresponding scale information by adopting a residual grouping convolution unit.
For example: the downsampling operation is carried out through the convolution with the kernel size of 2 multiplied by 2 and the step length of 2, the high semantic features are obtained through three times of downsampling, and in the process, because the added parameters and the calculated amount of residual error grouping convolution are reduced.
S103: low semantic features are extracted from the image representation of the multi-scale information and combined with the high semantic features respectively.
Directly connecting these high and low semantic feature maps without weighing their importance is not the best way to efficiently integrate them. In fact, the multi-level features may not be applicable to all types of input images, which may lead to information redundancy and thus to erroneous segmentation of the tumor. In addition, in the tumor image, the tumor features are only related to the visual feature information and the context information of the local area. This is similar to the phenomenon of human vision that only focuses on key local features when observing a thing.
Wherein, the low semantic features and the high semantic features are combined by adopting a jump connection structure.
S104: and respectively carrying out weighted recalibration on the space and channel dimensions of the combined features based on a space-channel attention mechanism to obtain corresponding attention features.
And respectively carrying out weighted recalibration on the space and channel dimensions of the combined features by using a space-channel attention module to obtain corresponding attention features.
Specifically, the process of obtaining the corresponding attention feature is as follows:
s1041: globally pooling width and height of the combined features, aggregating spatial information of the features using average pooling and maximum pooling;
s1042: the characteristics of the aggregated spatial information are subjected to a multilayer perceptron, and then multiplied by the characteristics of the original combination to obtain the channel attention characteristics;
s1043: compressing the channel in the channel attention feature, extracting the maximum value, and then multiplying the maximum value by the channel attention feature to obtain the final attention feature.
S105: and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
In the embodiment, the characteristics of the brain tumor image can be fully extracted from the multi-scale information of the brain tumor image, and then the brain tumor image is respectively subjected to coding operation based on the multi-scale information so as to extract the high semantic characteristics corresponding to the scale information and generate the image representation corresponding to the scale information; extracting low semantic features from the image representation of the multi-scale information, and respectively combining with the high semantic features; and respectively carrying out weighted recalibration on the space and channel dimensions of the combined features based on a space-channel attention mechanism to obtain corresponding attention features, so that the tumor region is concerned more, the channel information and the space information of the target are concerned more, and irrelevant regions are inhibited to improve the segmentation accuracy.
Example two
Referring to fig. 2, the present embodiment provides a system for brain tumor image segmentation, which specifically includes the following modules:
a multi-scale information extraction module 201, configured to extract multi-scale information of a brain tumor image;
an information encoding module 202, configured to perform an encoding operation on the brain tumor images respectively based on the multi-scale information to extract high semantic features corresponding to the scale information and generate an image representation corresponding to the scale information;
a feature combination module 203 for extracting low semantic features from the image representation of the multi-scale information, respectively combined with the high semantic features;
an attention feature determination module 204, configured to perform weighted recalibration on the combined features in space and channel dimensions respectively based on a space-channel attention mechanism, so as to obtain corresponding attention features;
and an image segmentation module 205, configured to restore the attention features to an original resolution, and obtain a brain tumor image segmentation result through feature representation of the image.
It should be noted that, each module in the present embodiment corresponds to each step in the first embodiment one to one, and the specific implementation process is the same, which is not described herein again.
EXAMPLE III
Referring to fig. 3, the present embodiment provides a system for brain tumor image segmentation, which specifically includes:
an image acquisition unit 301 for acquiring a brain tumor image;
an image segmentation unit 301, configured to obtain a brain tumor image segmentation result based on the brain tumor image and a pre-trained brain tumor image segmentation model;
the brain tumor image segmentation model mainly comprises four stages: the method comprises a data preprocessing stage, a network building stage, a model training stage and a model testing stage.
The experiment adopts a BraTS2019 brain tumor challenge match data set; the training set contained 335 cases, of which 256 patients were HGG and 75 patients were LGG; the test set included 166 cases; each case contains four modalities: t1, t2, t1ce, Flair. A data preprocessing stage: firstly, bias field correction is carried out on each mode of an MRI image by using an N4ITK algorithm, and the nonuniformity of the image is corrected. Each mode of the data was then normalized to zero mean and unit variance, and data amplification was performed using random inversion.
Building a residual grouping convolution U-Net based on a convolution block attention mechanism, as shown in FIG. 5, and a residual grouping convolution network training flow based on convolution block attention is shown in FIG. 4, wherein the model mainly comprises four parts:
a multi-scale input module 401 for extracting multi-scale information of the brain tumor image;
wherein, the input image with multi-scale size is obtained by using the maximal pooling operation with convolution kernel of 2 × 2 and step size of 2, and the multi-scale input module is formed.
An encoder module 402 for performing an encoding operation on the brain tumor images, respectively, based on the multi-scale information, to extract high semantic features corresponding to the scale information and generate an image representation corresponding to the scale information;
the encoder-decoder structure based on U-Net is improved, and the encoder part is used for extracting high semantic features of the tumor image and downsampling the image to reduce the resolution and the operation amount of the image. We implement the downsampling operation using a normal convolution with a convolution kernel of 2 x 2 and step size of 2 as shown in fig. 6(a) instead of the maximum pooling of the conventional U-Net. In addition to this, we use the residual packet convolution unit shown in fig. 6(c) instead of the residual convolution unit shown in fig. 6(b) in the U-Net network.
The method can increase the diagonal correlation between convolution kernels, pays attention to the local correlation between channels, only needs to carry out one-time 3 x 3 convolution integrally, then carries out channel segmentation without respectively carrying out 3 x 3 convolution, and is not easy to overfit; and the training and convergence speed of the model can be accelerated under the condition of reducing parameters and calculation amount. In each residual error grouping convolution unit, 8 branch convolution channels are arranged, each branch comprises three convolution layers, the sizes of convolution kernels are 1 multiplied by 1, 3 multiplied by 3 and 1 multiplied by 1, and each convolution unit is followed by a batch normalization layer and a random leakage ReLU activation function instead of the ReLU activation function used in the original U-Net. The input image is input into a decoder to construct segmentation after being subjected to feature extraction by the decoder module.
A skip join module 403 for joining the high semantic features extracted by the encoder module and the low semantic features extracted by the decoder module from the image representation of the multi-scale information, resulting in combined features;
a decoder module 404, in which a space-channel attention mechanism is embedded, and the combined features are respectively subjected to weighted recalibration of space and channel dimensions to obtain corresponding attention features; and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
The decoder module receives the image features from the encoder module and uses the transposed convolution to implement the upsampling operation to restore the features to the original resolution. In order to combine the feature maps in the decoder module with the corresponding feature maps of the decoder module, the original U-Net uses a hopping connection. However, directly connecting these high-semantic and low-semantic feature maps without weighing their importance is not the best way to efficiently integrate them. In fact, the multi-level features may not be applicable to all types of input images, which may lead to information redundancy and thus to erroneous segmentation of the tumor.
In addition, in the tumor image, the tumor features are only related to the visual feature information and the context information of the local area. This is similar to the phenomenon of human vision that only focuses on key local features when observing a thing. For the situation, a convolution block attention module which gives attention to space and channels is embedded in a decoder module and is used for highlighting key features relevant to tumors and suppressing irrelevant background features.
After the feature mapping extracted by the encoder is combined with the feature mapping obtained by the up-sampling, the attention module firstly carries out weighted recalibration on the combined feature mapping in space and channel dimensions, so that the target tumor is concerned more. The features processed by the attention module are input to a subsequent residual block convolution unit and a 2 × 2 transposed convolution for a subsequent upsampling operation. Fig. 7 shows the structure of the attention module. This module is located behind the three hop connections. Input features are first passed through an attention module, globally pooling features width and height, compressing them in spatial dimensions, and aggregating spatial information of features using average pooling and maximum pooling. And multiplying the data by the original characteristic points after passing through a multilayer perceptron to obtain the attention of the channel. Then the signal is input into a spatial attention unit to compress the channel and extract the maximum value. And then multiplying the channel attention feature points to obtain the final attention feature. In fig. 7, the average pooling layer, the maximum pooling layer, the multilayer perceptron, Sigmoid normalization, and the convolution are performed.
In the model training process, an Adam optimizer is adopted to optimize network parameters, set various learning rates, adopt adaptive loss functions Tverse local adaptive weighting and other strategy optimization models, promote the training convergence of the models and ensure that accurate and robust training results are obtained. And inputting the verification set of BRATS2019 into a trained brain tumor segmentation model. And (4) outputting the tumor target position and shape of the MRI brain tumor image through segmentation to obtain a segmentation result.
The overall performance of the segmentation model of the present embodiment depends not only on the structure of the network, but also on the choice of the loss function, especially in the presence of a high class imbalance problem. The brain tumor segmentation task has an inherent class imbalance problem due to the distribution of tumor and non-tumor regions. Unbalanced data may converge to local minima of suboptimal loss functions during the learning process, and predictions may strongly favor non-tumor tissue. Therefore, the loss function widely used in segmentation tasks is not suitable for our network training. With these functions, the network attempts to learn larger classes, resulting in poor segmentation performance. To solve this problem, we train the entire network using the Tvery Focal local loss function. The loss function can balance the categories through self-adaptive weighting, so that the convergence speed is accelerated, the segmentation performance is effectively improved, and the robustness is better under the condition of unbalanced categories.
The residual error grouping convolution unit of the embodiment can reduce parameters and calculated amount, quicken training and convergence and avoid overfitting. The high-level feature map and the low-level feature map are fused by a jump connection. An attention mechanism is introduced into a decoder module, so that the features in two dimensions of space and channels can be weighted, network confusion of multi-stage features in the fusion process is avoided, tumor-related features are highlighted, and irrelevant background features are inhibited. By means of the residual grouping convolution U-Net based on the attention mechanism, high segmentation efficiency is obtained, meanwhile, high precision and robustness are guaranteed, and therefore accurate image segmentation is guaranteed to assist a doctor in diagnosis and treatment.
Example four
The present embodiment provides a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for brain tumor image segmentation as described above.
EXAMPLE five
The present embodiment provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for brain tumor image segmentation as described above when executing the program.
Referring to fig. 8, a structural diagram of the electronic device in this embodiment is shown. It should be noted that the electronic device 500 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiments of the present invention.
As shown in fig. 8, the electronic apparatus 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for system operation are also stored. The central processing unit 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a Local Area Network (LAN) card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program, when executed by the central processing unit 501, performs various functions defined in the apparatus of the present application.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for brain tumor image segmentation, comprising:
extracting multi-scale information of the brain tumor image;
respectively carrying out coding operation on the brain tumor image based on the multi-scale information so as to extract high semantic features corresponding to the scale information and generate image representation corresponding to the scale information;
extracting low semantic features from the image representation of the multi-scale information, and respectively combining with the high semantic features;
respectively carrying out weighted recalibration on the combined features according to a space-channel attention mechanism to obtain corresponding attention features;
and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
2. The method of claim 1, wherein the combined features are separately weighted and re-scaled in space and channel dimensions using a space-channel attention module to obtain corresponding attention features.
3. The method for brain tumor image segmentation as set forth in claim 1, wherein the process of obtaining the corresponding attention feature is:
globally pooling width and height of the combined features, aggregating spatial information of the features using average pooling and maximum pooling;
the characteristics of the aggregated spatial information are subjected to a multilayer perceptron, and then multiplied by the characteristics of the original combination to obtain the channel attention characteristics;
compressing the channel in the channel attention feature, extracting the maximum value, and then multiplying the maximum value by the channel attention feature to obtain the final attention feature.
4. The method of claim 1, wherein the encoding operation is performed on the brain tumor image using an encoder.
5. The method for brain tumor image segmentation as claimed in claim 1 wherein a residual block convolution unit is employed to extract high semantic features corresponding to scale information.
6. The method for brain tumor image segmentation as set forth in claim 1, wherein the low semantic features and the high semantic features are combined using a skip join structure.
7. A system for brain tumor image segmentation, comprising:
the multi-scale information extraction module is used for extracting multi-scale information of the brain tumor image;
the information coding module is used for respectively carrying out coding operation on the brain tumor images based on the multi-scale information so as to extract high semantic features corresponding to the scale information and generate image representations corresponding to the scale information;
a feature combination module for extracting low semantic features from the image representation of the multi-scale information, respectively combined with the high semantic features;
the attention characteristic determination module is used for respectively carrying out weighted recalibration on the space and channel dimensions of the combined characteristics based on a space-channel attention mechanism to obtain corresponding attention characteristics;
and the image segmentation module is used for restoring the attention characteristics to the original resolution and obtaining a brain tumor image segmentation result through the characteristic representation of the image.
8. A system for brain tumor image segmentation, comprising:
an image acquisition unit for acquiring a brain tumor image;
the image segmentation unit is used for obtaining a brain tumor image segmentation result based on the brain tumor image and a brain tumor image segmentation model trained in advance;
wherein the brain tumor image segmentation model comprises:
a multi-scale input module for extracting multi-scale information of the brain tumor image;
an encoder module for performing encoding operations on brain tumor images, respectively, based on the multi-scale information, to extract high semantic features corresponding to the scale information and generate image representations corresponding to the scale information;
a skip connection module for connecting the high semantic features extracted by the encoder module and the low semantic features extracted by the decoder module from the image representation of the multi-scale information to obtain combined features;
a decoder module, in which a space-channel attention mechanism is embedded, and the combined features are respectively weighted and re-calibrated in space and channel dimensions to obtain corresponding attention features; and restoring the attention features to the original resolution, and obtaining a brain tumor image segmentation result through the feature representation of the image.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for brain tumor image segmentation of any one of claims 1 to 6.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps in the method for brain tumor image segmentation according to any one of claims 1 to 6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111178058.1A CN114066908A (en) | 2021-10-09 | 2021-10-09 | Method and system for brain tumor image segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111178058.1A CN114066908A (en) | 2021-10-09 | 2021-10-09 | Method and system for brain tumor image segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114066908A true CN114066908A (en) | 2022-02-18 |
Family
ID=80234489
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111178058.1A Pending CN114066908A (en) | 2021-10-09 | 2021-10-09 | Method and system for brain tumor image segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114066908A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115312198A (en) * | 2022-06-29 | 2022-11-08 | 河南省人民医院 | Deep learning brain tumor prognosis analysis modeling method and system combining attention mechanism and multi-scale feature mining |
| CN117409014A (en) * | 2023-10-09 | 2024-01-16 | 齐鲁工业大学(山东省科学院) | Lightweight heart MRI segmentation method based on feature selection |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110533045A (en) * | 2019-07-31 | 2019-12-03 | 中国民航大学 | A kind of luggage X-ray contraband image, semantic dividing method of combination attention mechanism |
| CN111192245A (en) * | 2019-12-26 | 2020-05-22 | 河南工业大学 | Brain tumor segmentation network and method based on U-Net network |
| CN112287940A (en) * | 2020-10-30 | 2021-01-29 | 西安工程大学 | Semantic segmentation method of attention mechanism based on deep learning |
| CN112365496A (en) * | 2020-12-02 | 2021-02-12 | 中北大学 | Multi-modal MR image brain tumor segmentation method based on deep learning and multi-guidance |
| CN113344951A (en) * | 2021-05-21 | 2021-09-03 | 北京工业大学 | Liver segment segmentation method based on boundary perception and dual attention guidance |
-
2021
- 2021-10-09 CN CN202111178058.1A patent/CN114066908A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110533045A (en) * | 2019-07-31 | 2019-12-03 | 中国民航大学 | A kind of luggage X-ray contraband image, semantic dividing method of combination attention mechanism |
| CN111192245A (en) * | 2019-12-26 | 2020-05-22 | 河南工业大学 | Brain tumor segmentation network and method based on U-Net network |
| CN112287940A (en) * | 2020-10-30 | 2021-01-29 | 西安工程大学 | Semantic segmentation method of attention mechanism based on deep learning |
| CN112365496A (en) * | 2020-12-02 | 2021-02-12 | 中北大学 | Multi-modal MR image brain tumor segmentation method based on deep learning and multi-guidance |
| CN113344951A (en) * | 2021-05-21 | 2021-09-03 | 北京工业大学 | Liver segment segmentation method based on boundary perception and dual attention guidance |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115312198A (en) * | 2022-06-29 | 2022-11-08 | 河南省人民医院 | Deep learning brain tumor prognosis analysis modeling method and system combining attention mechanism and multi-scale feature mining |
| CN115312198B (en) * | 2022-06-29 | 2023-09-19 | 河南省人民医院 | Deep learning brain tumor prognosis analysis modeling method and system combining attention mechanism and multi-scale feature mining |
| CN117409014A (en) * | 2023-10-09 | 2024-01-16 | 齐鲁工业大学(山东省科学院) | Lightweight heart MRI segmentation method based on feature selection |
| CN117409014B (en) * | 2023-10-09 | 2024-05-28 | 齐鲁工业大学(山东省科学院) | Lightweight heart MRI segmentation method based on feature selection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784671B (en) | Pathological image focus region detection method based on multi-scale deep learning | |
| CN115661144B (en) | Adaptive medical image segmentation method based on deformable U-Net | |
| US20230281809A1 (en) | Connected machine-learning models with joint training for lesion detection | |
| CN114581662B (en) | Brain tumor image segmentation method, system, device and storage medium | |
| CN109816666B (en) | Symmetrical full convolution neural network model construction method, fundus image blood vessel segmentation device, computer equipment and storage medium | |
| CN114066908A (en) | Method and system for brain tumor image segmentation | |
| CN112862805B (en) | Automatic auditory neuroma image segmentation method and system | |
| CN111583262A (en) | Blood vessel segmentation method and system | |
| WO2021136368A1 (en) | Method and apparatus for automatically detecting pectoralis major region in molybdenum target image | |
| CN112949654A (en) | Image detection method and related device and equipment | |
| Osadebey et al. | Three-stage segmentation of lung region from CT images using deep neural networks | |
| CN115375711A (en) | Image segmentation method of global context attention network based on multi-scale fusion | |
| CN113223015A (en) | Vascular wall image segmentation method, device, computer equipment and storage medium | |
| CN114418987A (en) | Retinal vessel segmentation method and system based on multi-stage feature fusion | |
| Sander et al. | Autoencoding low-resolution MRI for semantically smooth interpolation of anisotropic MRI | |
| Lim et al. | Motion artifact correction in fetal MRI based on a Generative Adversarial network method | |
| CN116030063A (en) | Classification diagnosis system, method, electronic device and medium for MRI image | |
| CN114049315B (en) | Joint recognition method, electronic device, storage medium, and computer program product | |
| CN113379770B (en) | Construction method of nasopharyngeal carcinoma MR image segmentation network, image segmentation method and device | |
| CN113177938B (en) | Method and device for segmenting brain glioma based on circular convolution kernel and related components | |
| CN111462004B (en) | Image enhancement method and device, computer equipment and storage medium | |
| CN113888556A (en) | Retinal blood vessel image segmentation method and system based on differential attention | |
| CN112949585A (en) | Identification method and device for blood vessels of fundus image, electronic equipment and storage medium | |
| CN116797615A (en) | Method for segmenting lesion region in clinical image based on frequency domain supervision and dynamic convolution | |
| CN115760651A (en) | Improved RegGAN low-dose CT image denoising method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |