CN117392137B - Intracranial aneurysm image detection method, system, equipment and medium - Google Patents
Intracranial aneurysm image detection method, system, equipment and medium Download PDFInfo
- Publication number
- CN117392137B CN117392137B CN202311703651.2A CN202311703651A CN117392137B CN 117392137 B CN117392137 B CN 117392137B CN 202311703651 A CN202311703651 A CN 202311703651A CN 117392137 B CN117392137 B CN 117392137B
- Authority
- CN
- China
- Prior art keywords
- module
- image
- loss
- intracranial aneurysm
- mlp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 114
- 201000008450 Intracranial aneurysm Diseases 0.000 title claims abstract description 87
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000000605 extraction Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 16
- 230000011218 segmentation Effects 0.000 claims description 14
- 238000007781 pre-processing Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 10
- 208000011829 combined pulmonary fibrosis-emphysema syndrome Diseases 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 2
- 210000002569 neuron Anatomy 0.000 claims description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 claims 1
- 238000004364 calculation method Methods 0.000 abstract description 4
- 230000008447 perception Effects 0.000 abstract description 3
- 206010002329 Aneurysm Diseases 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 5
- 238000003745 diagnosis Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012952 Resampling Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 238000002583 angiography Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002490 cerebral effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004195 computer-aided diagnosis Methods 0.000 description 1
- 239000002872 contrast media Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005489 elastic deformation Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000474 nursing effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000002792 vascular Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention belongs to the technical field of medical image processing, and particularly discloses a method, a system, equipment and a medium for detecting intracranial aneurysm images. The invention improves the extraction of the deep context feature of the image, the edge of the shallow layer of the image, the outline and other feature information by introducing the context perception pyramid feature extraction module, the channel attention module and the multi-scale convolution module on the basis of the 3D U-Net model, provides more depth features with channel weight information for the decoder end, improves the global learning capacity of the model, is more suitable for the detection task of the small target of the intracranial aneurysm, and effectively improves the detection performance of the intracranial aneurysm. The lightweight 3D MAU-Net detection model based on the multi-layer perceptron provided by the invention has fewer parameters, higher training speed and lower calculation complexity on the premise of ensuring that the detection performance is not obviously affected.
Description
Technical Field
The invention belongs to the technical field of medical image processing, and particularly relates to a method, a system, equipment and a medium for detecting intracranial aneurysm images.
Background
Intracranial aneurysms have high mortality due to their special location of incidence, and early discovery of uncracked intracranial aneurysms is the best means to reduce mortality. At present, the 3D TOF-MRA (Three Dimensional Timeof Flight Magnetic Resonance Angiography, three-dimensional time-leaping method magnetic resonance vascular imaging) technology has great application potential in intracranial aneurysm examination and diagnosis, does not use contrast agent, and has the advantages of safety and noninvasive property. However, when the 3D TOF-MRA image is used for aneurysm detection, the problems of insensitivity to small intracranial aneurysms, long training time, high memory occupation and the like exist, meanwhile, the number of layers of the 3D MRA image is large, the manual film reading workload of doctors is greatly increased, and the sensitivity of the diagnosis of radiologists can be reduced due to high-intensity film reading work, so that a series of problems such as missed diagnosis and misdiagnosis are caused. The MIP (Maximum Intensity Projection ) image generated by the rotation projection of the 3D TOF-MRA image largely retains the density information of the original image, but the quality and reliability of the MIP image mainly depend on the quality and parameter selection of the original data, which requires professional skills and experience, and if the parameter selection is poor, the acquired related information amount is more limited.
In recent years, deep learning models such as Convolutional Neural Networks (CNNs), full Convolutional Networks (FCNs), and the like have exhibited excellent performance in medical image detection and segmentation. For the study of the deep learning algorithm of the intracranial aneurysm, the network model is constructed as a primary factor considered for improving the detection performance of the intracranial aneurysm, because in the study of the intracranial aneurysm on a 3D image, the algorithm is realized by adopting a basic structure of an encoder and a decoder, and the improvement is usually developed around the thought of adjusting the network structure, modifying the size of a convolution kernel, extracting and fusing multi-scale features, adding an attention mechanism, improving a loss function and the like, so that the network is helped to learn the depth features with finer granularity through the improvement, and the detection performance of the algorithm is improved. In addition, part of researchers also combine the traditional algorithm with the deep learning algorithm, and based on the original deep learning model, the traditional algorithm is embedded into the whole detection flow in a pre-processing or post-processing mode, so that the detection precision of the algorithm is improved. Still other researchers combine two or more lightweight deep learning models and propose a multi-stage based learning strategy to achieve fine segmentation and detection of intracranial aneurysms.
In conclusion, the appearance of computer-aided diagnosis provides an important reference for doctor diagnosis, and the existing intracranial aneurysm aided diagnosis technology is not mature enough, and has the problems of long detection time, insufficient accuracy and the like.
Researchers have made a lot of effort to improve the sensitivity and accuracy of intracranial aneurysm detection models. The detection model is designed based on a convolution network, so that the detection sensitivity of the detection model to the intracranial aneurysm is very high, an encoder and decoder structure similar to the Unet is proposed by Geng et al, and the detection model has higher detection rate to the intracranial aneurysm of the 3D TOF-MRA image, and is suitable for clinical screening of the intracranial aneurysm. Claux et al propose a two-stage Unet network detection model that concatenates two Unets, wherein the first Unet model is used to segment the cerebral vessels and generate a binary image, and then the segmented binary image is sent to the second Unet model for training, finally, the detection of intracranial aneurysms is realized, the sensitivity of the algorithm reaches 78%, and the recognition rate of positive samples reaches 62%. Although the deep learning method has higher detection accuracy, it also brings more calculation cost, and this defect is more remarkable in the case of 3D medical images.
Disclosure of Invention
The invention aims to provide an intracranial aneurysm image detection method, which utilizes a 3D MAU-Net detection model (3D Multi-scale attention U-Net, namely a Multi-scale attention detection model) to perform feature extraction on an original 3D TOF-MRA image, so that original features of the image are reserved to the greatest extent, the model is enabled to pay more attention to segmentation of small targets, the accuracy of intracranial aneurysm detection and segmentation is improved, and a linear network is used for replacing part of convolution modules of the original model, so that the complexity of the model is reduced and the training time of the model is shortened on the premise that the detection performance of the model is basically unchanged.
In order to achieve the above purpose, the invention adopts the following technical scheme:
an intracranial aneurysm image detection method comprises the following steps:
step 1, preprocessing original 3D TOF-MRA;
and 2, building a 3D MAU-Net detection model, sending the preprocessed image into the 3D MAU-Net detection model, and detecting the intracranial aneurysm of the image to obtain a segmentation result of the intracranial aneurysm.
In addition, on the basis of the intracranial aneurysm image detection method, the invention also provides an intracranial aneurysm image detection system which is suitable for the intracranial aneurysm image detection method, and the intracranial aneurysm image detection system adopts the following technical scheme:
an intracranial aneurysm image detection system comprising:
the preprocessing module is used for preprocessing the original 3D TOF-MRA;
and the intracranial aneurysm identification module is used for building a 3D MAU-Net detection model, sending the preprocessed image into the 3D MAU-Net detection model, and carrying out intracranial aneurysm detection on the image to obtain an intracranial aneurysm segmentation result.
In addition, on the basis of the intracranial aneurysm image detection method, the invention further provides computer equipment which comprises a memory and one or more processors.
The memory stores executable codes, and the processor is used for realizing the steps of the intracranial aneurysm image detection method when executing the executable codes.
Furthermore, on the basis of the intracranial aneurysm image detection method, the invention further provides a computer readable storage medium on which a program is stored.
The program, when executed by a processor, is configured to implement the steps of the intracranial aneurysm image detection method described above.
The invention has the following advantages:
the invention provides a 3D MAU-Net detection model, which is obtained by introducing a context perception pyramid feature extraction module, a channel attention module and a multi-scale convolution module on the basis of a 3D U-Net model, so that the extraction of feature information such as deep image context features, edges of shallow images and contours is deepened, more depth features with channel weight information are provided for a decoder end, the global learning capacity of the model is improved, the model is more suitable for intracranial aneurysm small target detection tasks, and the detection performance of intracranial aneurysms is effectively improved. Aiming at the common problems of the U-Net network, namely the characteristics of overlarge parameter quantity, overlong model training time and overhigh requirement on computer performance, and can not be well applied to the hospital nursing point for rapid image segmentation, the lightweight 3D MAU-Net detection model based on the multi-layer perceptron has fewer parameters, higher training speed and lower calculation complexity on the premise of ensuring that the detection performance is not obviously influenced.
Drawings
FIG. 1 is a flow chart of an automatic intracranial aneurysm detection model based on an attention mechanism and a multi-layer perceptron in an embodiment of the invention.
Fig. 2 is a schematic diagram of a network architecture of a 3D MAU-Net detection model constructed in an embodiment of the present invention.
Fig. 3 is a schematic diagram of a PCA module according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of hole convolution in an embodiment of the present invention.
Fig. 5 is a schematic diagram of an MC module according to an embodiment of the invention.
FIG. 6 is a schematic diagram of a 3D Tokenized MLP in an embodiment of the invention.
Fig. 7 is a schematic diagram of a deep convolution in an embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the attached drawings and detailed description:
example 1
In this embodiment 1, an intracranial aneurysm image detection method is described, which proposes feature extraction of an original 3D TOF-MRA image, so as to preserve original features of the image to the greatest extent, and improve detection performance of the intracranial aneurysm. And the linear network is used for replacing part of convolution modules in the original model, so that the complexity of the model is reduced and the training time of the model is shortened on the premise of ensuring that the detection performance of the model is basically unchanged. Therefore, the invention provides an intracranial aneurysm automatic detection model based on an attention mechanism and a Multi-layer perceptron, firstly, a PCA module (Pyramid channel attention) and a MC module (Multi-scale convolution) are added in 3D U-Net to deeply extract features of different layers, then, a decoder is used for carrying out feature fusion among different layers, and finally, a final intracranial aneurysm detection result is obtained through deep supervision and guidance network training. Based on the model, the invention proposes to replace the original partial convolution module by a three-dimensional labeling multi-layer perceptron module (3D Tokenized MLP) so as to reduce the complexity of the model.
As shown in fig. 1, the intracranial aneurysm image detection method in the embodiment includes the following steps:
and step 1, preprocessing the original 3D TOF-MRA.
The preprocessing operations include resampling, data normalization, and random data enhancement.
Resampling is to normalize voxels of different sizes in the original image to the same size, setting the median pixel pitch to [0.7,0.43,0.43].
The data normalization is to scale data with different sizes into the same data interval through processing, and the calculation formula is as follows:
，；
wherein the method comprises the steps ofIs the standard deviation of the two-dimensional image,as an average value of the values,a voxel value representing each point is represented,representing the normalized score.
Random data enhancement: when the algorithm performs training experiments, random data enhancement can perform random elastic deformation, random rotation and random scale transformation on the data in real time.
And 2, building a 3D MAU-Net detection model, sending the preprocessed image into the 3D MAU-Net detection model, and detecting the intracranial aneurysm of the image to obtain a segmentation result of the intracranial aneurysm.
The original TOF-MRA image is sent to the 3D MAU-Net detection model to obtain a segmentation result, so that the original characteristics of the image can be fully learned, and the detection performance of intracranial aneurysms is effectively improved.
The invention provides an intracranial aneurysm automatic detection model combining an attention mechanism with a 3D U-Net model in deep learning, and provides a pyramid channel attention module, a multi-scale convolution module and a 3D Tokenized MLP module.
Specifically, the 3D MAU-Net detection model is based on the traditional 3D U-Net model, a context awareness pyramid feature extraction module CPFE, a channel attention module CA and a multi-scale convolution module MC are introduced, extraction of feature information of deep context features, edges and contours of images is achieved, and in addition, the invention further provides that a three-dimensional labeling MLP (namely a 3D processed MLP) module is used for replacing a part of convolution module, so that the complexity of the 3D MAU-Net detection model is reduced fully on the premise that the detection performance is basically unchanged. The method can solve the problems that the existing manual interpretation method is easily influenced by subjective experience of doctors and wastes time and labor, and further improves the accuracy and efficiency of intracranial aneurysm detection.
After the step 1 is completed, the intracranial aneurysm image is pre-cut, and then is sent into an automatic intracranial aneurysm detection model based on an attention mechanism and a multi-layer perceptron, namely a 3D MAU-Net detection model. The detection model takes a 3D U-Net structure as a main network, is improved by introducing a PCA and MC functional module and a 3D enabled MLP module, so that the detection model is suitable for intracranial aneurysm detection, and the network architecture of the 3D MAU-Net detection model is shown as a figure 2 and comprises an encoder, a PCA module, an MC module, a decoder, a 3D enabled MLP module and a depth supervision module;
the processing flow of the preprocessed 3D TOF-MRA image in the 3D MAU-Net detection model is as follows:
firstly, sending the preprocessed 3D TOF-MRA image to an encoder, performing common convolution and residual connection to extract image features, and inputting the features extracted by the encoder to a 3D Tokenized MLP module to perform shifting MLP operation;
the encoder outputs five layers of features f1, f2, f3, f4 and f5 together with the 3D Tokenized MLP;
the advanced features f1 and f2 are subjected to feature learning through the MC module, extraction of image edge information is enhanced, and feature images of f1 'and f 2' are output; extracting the advanced features f3, f4 and f5 by using a PCA module, outputting f3 ', f4 ' and f5 ' feature images to obtain more complete bottom features, and then passing through a 3D enabled MLP module and a decoder module;
and f3 ' feature images and the upper layer S4 of the decoder are fused and input into the third layer decoder to obtain S3, f2 ' feature images and the upper layer S3 of the decoder are fused and input into the second layer decoder to obtain S2, f1 ' feature images and the upper layer S2 of the decoder are fused and input into the third layer decoder to obtain S1, the upper three layers of the decoder output S1, S2 and S3 are guided by a mixed loss function to train through a depth supervision module, and a final intracranial aneurysm detection result is obtained.
The encoder is formed by pooling two Conv3d convolution layers and a maximum value, the number of characteristic channels is doubled after each downsampling, and five layers of characteristics of outputs f1, f2, f3, f4 and f5 are activated through convolution and LeakyReLU.
Since the main improvement of 3D U-Net is based on feature fusion of the decoder, features extracted prior to entry into the decoder are critical for intracranial aneurysm detection. When the channel dimension performs feature learning, importance degrees of all channels are emphasized through different module structures, and feature extraction capacity of a deep learning network is improved at an encoder end, so that overall detection performance of a model is improved.
The PCA module structure is shown in fig. 3, the PCA module is composed of a context perception pyramid feature extraction module CPFE module and a channel attention module CA, the CPFE module firstly convolves a feature image from an encoder by utilizing cavities with convolution kernel sizes of 3 multiplied by 3 and different expansion rates (3, 5 and 7 respectively) to obtain multi-scale multi-receptive field features, then fuses the feature image obtained by the cavity convolutions with the feature image with the convolution kernel size of 1 multiplied by 1 to obtain an output feature layer P, then sends the fused feature image P into the CA module, gives different weights to different channels by utilizing the CA module, fully utilizes effective information, and finally inputs the output feature information into a decoder of the same layer. Wherein, definition of the hole convolution is as follows:
。
wherein the method comprises the steps ofIs an input signal which is provided with a signal,is an output signal which is provided with a signal,representing a length ofR corresponds to the samplingExpansion ratio of (2); in standard convolution, r=1.
Hole convolution as shown in FIG. 4, when the CA module assigns a larger weight to a channel that plays an important role in image detection, it is assumed thatFor PResetting, then transposing the reset P, and multiplying the transposed P and the reset P by matrixFinally, obtaining the channel attention map by using softmax：
。
Wherein,representing the effect of the ith channel on the jth channel; thereafter, matrix multiplication is performed between X transpose and reset of P, and the resulting result is reset to。
Finally, the result is combinedMultiplying and performing element-by-element summation operation on P to obtain final output：
。
Wherein,representing the output characteristics of the channel,the importance weight representing the ith feature dimension,a bias term representing a jth neuron;representing the scale parameters, the weights are learned gradually starting from 0.
The formula shows that the final feature of each channel is a weighted sum of the features of all channels and the original features, which is beneficial to improving the discrimination between the features.
As shown in fig. 5, the MC module performs a convolution operation on a feature map obtained at a certain time by using 3 x 3 convolution, 5 x 5 convolution, and 7 x 7 convolution, while simultaneously convolving with 1 x 1 convolution, and obtaining a new feature map with different feature information, and then fusing the different feature maps to obtain a final output feature map. The multi-scale convolution does not change the size of the original feature map, but only enriches the features of the image through convolution operations of different convolutions, and extracts interesting feature information from the image from a global view angle, so that the performance of the model is improved. Because the multi-scale convolution fully utilizes convolution kernels with different sizes, not only can abundant context information be obtained in the feature extraction process, but also the edge contour information of the image can be better reserved.
As shown in FIG. 6, the 3D Tokenized MLP is a three-dimensional extension of the 2D Tokenized MLP block. In the 3D Tokenized MLP module, the feature map is processed as follows:
transmitting the feature map to a first Shifted multi-layer perceptron (Shifted MLP) to carry out width-wise shifting MLP operation, and carrying out depth convolution (DWConv) on the feature information after shifting; after a depth convolution, using a GELU activation function to output to the following Shifted MLP, respectively performing a shifting MLP operation on the height and a shifting MLP operation on the depth on the second shifting multi-layer perceptron Shifted MLP and the third shifting multi-layer perceptron Shifted MLP, performing the depth convolution again, and performing LN normalization on the feature map; here, the 3D token MLP uses a residual connection to linearly add the initial feature map token to the shifted MLP feature map and pass the output features to the next co-layer PCA module.
The mathematical computation process in the 3D Tokenized MLP module can be calculated as:
；
；
；
；
；
。
wherein X is shift Representing the output shifted in the X direction, Y shift Representing the output shifted in the Y direction, Z shift Representing the output shifted in the Z direction; token represents a marker location; t represents the original feature map token, H represents the height, W represents the width, D represents the depth, DWconv represents the depth convolution, LN represents the layer normalization, GELU is the activation function, X represents the output of the X-direction through the Tokenized MLP, Y represents the output of the Y-direction through the Tokenized MLP, and Z is the current output of the module.
When the depth convolution carries out convolution operation, the number of convolution kernels is consistent with the number of channels of the upper layer, and the number of output characteristic diagrams is ensured to be consistent with the number of input characteristic diagrams. As shown in fig. 7.
The decoder end is composed of functional modules, and the decoding stage and the encoding stage are mirror images.
In the decoding stage, the resolution is doubled once per upsampling. And the information of the encoding stage and the information of the decoding stage are combined through the jump connection, so that the image characteristic information quantity contained in the network is increased.
After the 3D MAU-Net detection model is built, the model needs to be trained, and the training process is as follows:
and sending the data set of the TOF-MRA image obtained after pretreatment into a 3D MAU-Net detection model for training.
The model complex training optimization algorithm adopts an ADAM algorithm with an initial learning rate lr of 0.001;
weight decay index of 10 -5 Setting a moving index average valueThe training loss is monitored every 30 iteration cycles as long asDoes not reduce by 5X 10 -4 Lr is reduced by a factor of 5; the network adopts a mixed loss function;
the number of the batch sizes is 30, the maximum iteration period of the network training is 300, each iteration period comprises 150 iterations, and when the iteration period reaches the maximum value or lr is less than 10 -8 The network stops training;
and (5) saving the model weight at the highest accuracy and lower loss under the verification set at present, and completing the training process.
In the training process of the 3D MAU-Net detection model, a Focal loss function and a Lovasz-Softmax loss function are adopted to jointly represent global loss and local loss, and the two losses are combined to be expressed as: loss=loss F +loss L 。
Wherein loss represents a loss function of the 3D MAU-Net detection model, loss F Table Focal loss, loss L Representing the lovassz-Softmax loss. Focal loss is used to deal with extreme foreground-background class imbalance problems, focusing loss on difficult-to-separate samples, starting from the difficulty of sample classification, defined as follows:
。
wherein y is i For input instance x i True category, p i To predict input instance x i Probability of belonging to real class, for extremely unbalanced class, addTo predict samples with low probability, and to switch positive samplesThe fluence becomes large, gamma is an adjustable factor, N represents the total number of samples, and alpha represents a weight parameter.
Lovasz-Softmax can promote the intersection ratio IoU score of the segmentation result, and IoU score is also called Jaccard index, which is defined as:。
wherein the method comprises the steps ofRepresenting standard results, y represents network prediction results, so Jaccard loss is defined as:
。
optimizing Jaccard loss by Lovasz-Softmax is often used to fill in the gaps in Jaccard loss partitioning, recover small objects, and form a more reasonable global partitioning, defined as follows:
；
。
wherein p represents the number of pixels;representing the first of the network predictions𝑖Value of individual element->Represents the first of the standard results𝑖The values of the individual elements.
Furthermore, in order to verify the effectiveness of the method of the invention, the following experiments are also presented:
the data used in this experiment was provided by published data in a hospital and MICCAI2020 aneurysm detection and segmentation (ADAM) challenge, and the experimental data were 1043 TOF-MRA images, 500 negative samples and 543 positive samples.
S1, preprocessing a data set:
the label used in the experiment is a binarized image, and since the negative sample does not have the original label, a label image which is consistent with the original image in size and has a voxel value of 0 needs to be generated in the process of constructing the data set. All subject data were then divided into large, medium, small and non-intracranial aneurysms patients, with intracranial aneurysms greater than 7mm in diameter labeled large aneurysms, medium aneurysms between 3mm and 7mm in diameter labeled medium aneurysms, small aneurysms less than 3mm in diameter labeled small aneurysms, and subjects not having an intracranial aneurysm labeled normal human specimens. And split it into training and testing sets.
S2, sending the training set of the TOF-MRA image obtained after pretreatment into a lightweight 3D MAU-Net detection model based on MLP for training.
The network training optimization algorithm provided by the model adopts an ADAM algorithm with initial learning rate of lr=0.001, and weight attenuation index is 10 -5 Setting a moving index average valueThe training loss is monitored every 30 iteration cycles as long asDoes not reduce by 5X 10 -4 Lr is reduced by a factor of 5. The network employs a hybrid loss function. The number of the batch sizes is 30, the maximum iteration period of the network training is 300, each iteration period comprises 150 iterations, and when the iteration period reaches the maximum value or lr is less than 10 -8 The network stops training. And saving the model weight at the time of highest accuracy and lower loss under the verification set.
And S3, sending the test set into a lightweight 3D MAU-Net to obtain an intracranial aneurysm detection result. And carrying out visualization processing on the image to obtain a visualization result. The detection model outputs intracranial aneurysm binarization images, the original MIP images and the corresponding binarization images are simultaneously opened by amide software to splice the images, and the aneurysm images are marked by different colors, so that the observation is convenient.
Example 2
This embodiment 2 describes an intracranial aneurysm image detection system based on the same inventive concept as the intracranial aneurysm image detection method described in embodiment 1 above.
Specifically, an intracranial aneurysm image detection system comprising:
the preprocessing module is used for preprocessing the original 3D TOF-MRA;
and the intracranial aneurysm identification module is used for building a 3D MAU-Net detection model, sending the preprocessed image into the 3D MAU-Net detection model, and carrying out intracranial aneurysm detection on the image to obtain an intracranial aneurysm segmentation result.
It should be noted that, in the intracranial aneurysm image detection system, the implementation process of the functions and roles of each functional module is specifically detailed in the implementation process of the corresponding steps in the method in the above embodiment 1, and will not be described herein.
Example 3
Embodiment 3 describes a computer apparatus for implementing the intracranial aneurysm image detection method described in embodiment 1.
In particular, the computer device includes a memory and one or more processors. Executable code is stored in the memory for implementing the steps of the intracranial aneurysm image detection method described above when the executable code is executed by the processor.
In this embodiment, the computer device is any device or apparatus having data processing capability, which is not described herein.
Example 4
Embodiment 4 describes a computer-readable storage medium for implementing the intracranial aneurysm image detection method described in embodiment 1.
Specifically, the computer-readable storage medium in this embodiment 4 has stored thereon a program for implementing the steps of the intracranial aneurysm image detection method described above when executed by a processor.
The computer readable storage medium may be an internal storage unit of any device or apparatus having data processing capability, such as a hard disk or a memory, or may be an external storage device of any device having data processing capability, such as a plug-in hard disk, a Smart Media Card (SMC), an SD Card, a Flash memory Card (Flash Card), or the like, which are provided on the device.
The foregoing description is, of course, merely illustrative of preferred embodiments of the present invention, and it should be understood that the present invention is not limited to the above-described embodiments, but is intended to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.
Claims (8)
1. An intracranial aneurysm image detection method is characterized by comprising the following steps:
step 1, preprocessing original 3D TOF-MRA;
step 2, building a 3D MAU-Net detection model, sending the preprocessed image into the 3D MAU-Net detection model, and detecting intracranial aneurysms on the image to obtain intracranial aneurysms segmentation results;
the built 3D MAU-Net detection model comprises an encoder, a PCA module, an MC module, a decoder, a 3D Tokenized MLP module and a depth supervision module;
the processing flow of the preprocessed 3D TOF-MRA image in the 3D MAU-Net detection model is as follows:
firstly, sending the preprocessed 3D TOF-MRA image to an encoder, performing common convolution and residual connection to extract image features, and inputting the features extracted by the encoder to a 3D Tokenized MLP module to perform shifting MLP operation;
the encoder outputs five layers of features f1, f2, f3, f4 and f5 together with the 3D Tokenized MLP;
the advanced features f1 and f2 are subjected to feature learning through the MC module, extraction of image edge information is enhanced, and feature images of f1 'and f 2' are output; extracting the advanced features f3, f4 and f5 by using a PCA module, outputting f3 ', f4 ' and f5 ' feature images to obtain more complete bottom features, and then passing through a 3D enabled MLP module and a decoder;
three layers of outputs S1, S2 and S3 on the decoder pass through a depth supervision module, and a mixed loss function guides a 3D MAU-Net detection model network to train so as to obtain a final intracranial aneurysm detection result;
the PCA module consists of a context awareness pyramid feature extraction module CPFE and a channel attention module CA; the CPFE module firstly convolves the feature map from the encoder by utilizing cavities with convolution kernel sizes of 3 multiplied by 3 and different expansion rates to obtain multi-scale multi-receptive field features, then fuses the feature map obtained by the cavity convolution with different expansion rates with the feature map with convolution kernel sizes of 1 multiplied by 1 to obtain an output feature layer P, then sends the fused feature map P into the CA module, gives different weights to different channels by utilizing the CA module, and finally inputs the output feature information into a decoder of the same layer.
2. The method for detecting an image of an intracranial aneurysm as claimed in claim 1,
the definition of the hole convolution is as follows:
wherein f [ i ]]Is an input signal g [ i ]]Is an output signal, h [ l ]]Representing a filter of length l, r corresponding to a sample f [ i ]]Expansion ratio of (2); suppose P ε R C×H×W R to P C×H Resetting, transposing the reset P, performing matrix multiplication on the transposed P and the reset P, and finally obtaining a channel attention map P epsilon R by using softmax C×C ：
Wherein x is ji Representing the effect of the ith channel on the jth channel; thereafter, matrix multiplication is performed between X transpose and reset of P, and the resulting result is reset to R C×H×W ；
Finally multiplying the result by betaAnd performs element-by-element summation operation on P to obtain final output A E R C×H×W ：
Wherein A is j Representing channel output characteristics, P i Importance weight, P, representing the ith feature dimension j A bias term representing a jth neuron; beta represents a scale parameter, and the weights are learned gradually from 0.
3. The method for detecting an image of an intracranial aneurysm as claimed in claim 1,
in a 3D processed MLP module, transmitting the feature map to a first Shifted MLP (multi-layer perceptron) to carry out width-wise shift MLP operation, and carrying out depth convolution DWConv on the feature information after shifting;
after a depth convolution, using a GELU activation function to output to the following Shifted MLP, respectively performing a shifting MLP operation on the height and a shifting MLP operation on the depth on the second shifting multi-layer perceptron Shifted MLP and the third shifting multi-layer perceptron Shifted MLP, performing the depth convolution again, and performing LN normalization on the feature map;
the 3D enabled MLP module uses a residual connection to linearly add the initial feature map token and the feature map after the shift MLP, and transmits the output features to the same-layer PCA module.
4. The method for detecting an image of an intracranial aneurysm as claimed in claim 1,
the training process of the 3D MAU-Net detection model is as follows:
sending the data set of the TOF-MRA image obtained after pretreatment into a 3D MAU-Net detection model for training;
wherein, the model complex training optimization algorithm adopts an ADAM algorithm with an initial learning rate lr of 0.001, and a weight attenuation index is 10 -5 Setting a moving index average valueMonitoring training loss once every 30 iteration cycles;
the network adopts a mixed loss function;
the number of the batch sizes is 30, the maximum iteration period of the network training is 300, each iteration period comprises 150 iterations, and when the iteration period reaches the maximum value or lr is less than 10 -8 The network stops training;
and saving the model weight to complete the training process.
5. The method for detecting an image of an intracranial aneurysm as claimed in claim 4,
in the training process of the 3D MAU-Net detection model, a Focal loss function and a Lovasz-Softmax loss function are adopted to jointly represent global loss and local loss, and the two losses are combined to be expressed as:
loss＝loss F +loss L ；
wherein loss represents a loss function of the 3D MAU-Net detection model; loss of loss F Represents a Focal loss, loss L Representing the Lovasz-Softmax loss;
loss F the definition is as follows:
wherein y is i For input instance x i True category, p i To predict input instance x i The probability of belonging to the true class of the class, for the problem of extremely unbalanced class, adds (1-p i ) γ Predicting samples with small probability, so that the attention of positive samples is increased, gamma is an adjustable factor, N represents the total number of samples, and alpha represents a weight parameter;
Lovasz-Softmax can promote the intersection ratio IoU score of the segmentation result, and IoU score is also called Jaccard index, which is defined as:
wherein y is * Representing standard results, y represents network prediction results, so Jaccard loss is defined as:
loss J ＝Δ J (y * ，y)＝1-J c (y * ，y)；
optimizing Jaccard loss by Lovasz-Softmax is often used to fill in the gaps in Jaccard loss partitioning, recover small objects, and form a more reasonable global partitioning, defined as follows:
m i ＝max(1-y i ·y i * ，0)；
wherein p represents the number of pixels; y is i Representing the value of the i-th element in the network prediction result, y i * Representing the value of the i-th element in the standard result.
6. An intracranial aneurysm image detection system for implementing the intracranial aneurysm image detection method as recited in any one of claims 1 to 5, the intracranial aneurysm image detection system comprising:
the preprocessing module is used for preprocessing the original 3D TOF-MRA;
and the intracranial aneurysm identification module is used for building a 3D MAU-Net detection model, sending the preprocessed image into the 3D MAU-Net detection model, and carrying out intracranial aneurysm detection on the image to obtain an intracranial aneurysm segmentation result.
7. A computer device comprising a memory and one or more processors, the memory having executable code stored therein, wherein the processor, when executing the executable code,
a step of implementing the intracranial aneurysm image detection method as recited in any one of claims 1 to 5.
8. A computer-readable storage medium having a program stored thereon, which when executed by a processor, implements the steps of the intracranial aneurysm image detection method as recited in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311703651.2A CN117392137B (en) | 2023-12-13 | 2023-12-13 | Intracranial aneurysm image detection method, system, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311703651.2A CN117392137B (en) | 2023-12-13 | 2023-12-13 | Intracranial aneurysm image detection method, system, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117392137A CN117392137A (en) | 2024-01-12 |
| CN117392137B true CN117392137B (en) | 2024-02-13 |
Family
ID=89468876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311703651.2A Active CN117392137B (en) | 2023-12-13 | 2023-12-13 | Intracranial aneurysm image detection method, system, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117392137B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021104056A1 (en) * | 2019-11-27 | 2021-06-03 | 中国科学院深圳先进技术研究院 | Automatic tumor segmentation system and method, and electronic device |
| CN113436166A (en) * | 2021-06-24 | 2021-09-24 | 深圳市铱硙医疗科技有限公司 | Intracranial aneurysm detection method and system based on magnetic resonance angiography data |
| CN113706451A (en) * | 2021-07-07 | 2021-11-26 | 杭州脉流科技有限公司 | Method, device, system and computer-readable storage medium for intracranial aneurysm identification detection |
| CN115147404A (en) * | 2022-08-06 | 2022-10-04 | 河南大学 | Intracranial aneurysm segmentation method with dual-feature fusion MRA image |
| CN115829914A (en) * | 2022-08-15 | 2023-03-21 | 首都医科大学 | Method for automatically identifying and segmenting intracranial aneurysm in 3.0T high-resolution MRI T1 sequence |
| CN115953393A (en) * | 2023-03-10 | 2023-04-11 | 济南市人民医院 | Intracranial aneurysm detection system, equipment and storage medium based on multitask learning |
| CN116823833A (en) * | 2023-08-30 | 2023-09-29 | 山东科技大学 | Method, system and equipment for detecting intracranial aneurysm by using omnibearing MIP image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11580646B2 (en) * | 2021-03-26 | 2023-02-14 | Nanjing University Of Posts And Telecommunications | Medical image segmentation method based on U-Net |
-
2023
- 2023-12-13 CN CN202311703651.2A patent/CN117392137B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021104056A1 (en) * | 2019-11-27 | 2021-06-03 | 中国科学院深圳先进技术研究院 | Automatic tumor segmentation system and method, and electronic device |
| CN113436166A (en) * | 2021-06-24 | 2021-09-24 | 深圳市铱硙医疗科技有限公司 | Intracranial aneurysm detection method and system based on magnetic resonance angiography data |
| CN113706451A (en) * | 2021-07-07 | 2021-11-26 | 杭州脉流科技有限公司 | Method, device, system and computer-readable storage medium for intracranial aneurysm identification detection |
| CN115147404A (en) * | 2022-08-06 | 2022-10-04 | 河南大学 | Intracranial aneurysm segmentation method with dual-feature fusion MRA image |
| CN115829914A (en) * | 2022-08-15 | 2023-03-21 | 首都医科大学 | Method for automatically identifying and segmenting intracranial aneurysm in 3.0T high-resolution MRI T1 sequence |
| CN115953393A (en) * | 2023-03-10 | 2023-04-11 | 济南市人民医院 | Intracranial aneurysm detection system, equipment and storage medium based on multitask learning |
| CN116823833A (en) * | 2023-08-30 | 2023-09-29 | 山东科技大学 | Method, system and equipment for detecting intracranial aneurysm by using omnibearing MIP image |
Non-Patent Citations (2)
| Title |
|---|
| 深度迭代融合的脑部磁共振图像颅骨去除网络;姚发展;李智;王丽会;程欣宇;张健;;中国图象图形学报;20201016(第10期);全文 * |
| 耿辰 ; 杨丽琴 ; 尹波 ; 李郁欣 ; 耿道颖 ; .基于深度学习的时间飞跃法磁共振血管成像对脑动脉瘤自动检测方法研究.上海医学.2020,(第05期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117392137A (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113077471B (en) | Medical image segmentation method based on U-shaped network | |
| CN112017189B (en) | Image segmentation method and device, computer equipment and storage medium | |
| CN111612754B (en) | MRI tumor optimization segmentation method and system based on multi-modal image fusion | |
| Wang et al. | FE-YOLOv5: Feature enhancement network based on YOLOv5 for small object detection | |
| CN113642390B (en) | Street view image semantic segmentation method based on local attention network | |
| CN114445670B (en) | Training method, device and equipment of image processing model and storage medium | |
| CN114494296A (en) | Brain glioma segmentation method and system based on fusion of Unet and Transformer | |
| CN113496494B (en) | Two-dimensional skeleton segmentation method and device based on DRR simulation data generation | |
| CN116309648A (en) | Medical image segmentation model construction method based on multi-attention fusion | |
| CN116935044B (en) | Endoscopic polyp segmentation method with multi-scale guidance and multi-level supervision | |
| CN112488996B (en) | Non-homogeneous three-dimensional esophageal cancer energy spectrum CT weak supervision automatic labeling method and system | |
| CN117422715A (en) | Global information-based breast ultrasonic tumor lesion area detection method | |
| CN117934824A (en) | Target region segmentation method and system for ultrasonic image and electronic equipment | |
| CN113706544A (en) | Medical image segmentation method based on complete attention convolution neural network | |
| CN117557774A (en) | Unmanned aerial vehicle image small target detection method based on improved YOLOv8 | |
| CN117252852A (en) | Lightweight skin lesion segmentation method and system based on convolutional neural network and multi-layer perceptron | |
| Qin et al. | Dynamically aggregating MLPs and CNNs for skin lesion segmentation with geometry regularization | |
| CN116823850A (en) | Cardiac MRI segmentation method and system based on U-Net and transducer fusion improvement | |
| CN118196628A (en) | Enhanced semantic-location feature fusion network method, device and application for extracting backbone based on different pre-training features | |
| Zheng et al. | Smaformer: Synergistic multi-attention transformer for medical image segmentation | |
| CN117392137B (en) | Intracranial aneurysm image detection method, system, equipment and medium | |
| Zhao et al. | VCMix-Net: A hybrid network for medical image segmentation | |
| CN115775252A (en) | Magnetic resonance image cervical cancer tumor segmentation method based on global local cascade | |
| Liu et al. | Medical Image Recognition Based on Multiscale Cascade Segmentation Network MCSnet | |
| CN117893934B (en) | Improved UNet3+ network unmanned aerial vehicle image railway track line detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |