CN117253045A - Hysteromyoma segmentation method and device of medical image, equipment and storage medium - Google Patents
Hysteromyoma segmentation method and device of medical image, equipment and storage medium Download PDFInfo
- Publication number
- CN117253045A CN117253045A CN202311336190.XA CN202311336190A CN117253045A CN 117253045 A CN117253045 A CN 117253045A CN 202311336190 A CN202311336190 A CN 202311336190A CN 117253045 A CN117253045 A CN 117253045A
- Authority
- CN
- China
- Prior art keywords
- convolution
- layer
- encoder
- medical image
- uterine fibroid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010046798 Uterine leiomyoma Diseases 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000011218 segmentation Effects 0.000 title claims abstract description 38
- 201000010260 leiomyoma Diseases 0.000 claims abstract description 45
- 208000010579 uterine corpus leiomyoma Diseases 0.000 claims abstract description 40
- 201000007954 uterine fibroid Diseases 0.000 claims abstract description 40
- 238000003709 image segmentation Methods 0.000 claims abstract description 24
- 238000011176 pooling Methods 0.000 claims abstract description 22
- 206010028980 Neoplasm Diseases 0.000 claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 14
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 16
- 238000005070 sampling Methods 0.000 claims description 14
- 230000004913 activation Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 230000009191 jumping Effects 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 9
- 238000002595 magnetic resonance imaging Methods 0.000 description 9
- 201000004458 Myoma Diseases 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 206010061692 Benign muscle neoplasm Diseases 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 101100295091 Arabidopsis thaliana NUDT14 gene Proteins 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 4
- 238000002604 ultrasonography Methods 0.000 description 4
- 238000002679 ablation Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 210000001519 tissue Anatomy 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 208000025421 tumor of uterus Diseases 0.000 description 2
- 206010046766 uterine cancer Diseases 0.000 description 2
- 210000004291 uterus Anatomy 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 206010028851 Necrosis Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000004071 biological effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000000338 in vitro Methods 0.000 description 1
- 230000009349 indirect transmission Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000002980 postoperative effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000004872 soft tissue Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image segmentation, and discloses a hysteromyoma segmentation method of medical images, which comprises the steps of acquiring a plurality of medical images for preprocessing to obtain a plurality of sample images to form a sample data set; constructing a U-shaped network model; training a U-shaped network model by using a sample data set to obtain a hysteromyoma image segmentation model with optimal model parameters; inputting the medical image to be detected into a uterine fibroid image segmentation model to obtain a tumor area corresponding to uterine fibroid; the U-shaped network model uses the Unet as a backbone network, combines a cavity convolution and an attention mechanism, suppresses unimportant features by applying a channel attention mechanism between each layer of the encoder, combines a cavity space convolution pooling pyramid with a convolution attention module at the bottom of the encoder, and can improve the recognition capability of the model on small-volume tumors, thereby improving the segmentation accuracy and reliability.
Description
Technical Field
The invention belongs to the technical field of image segmentation, and particularly relates to a method, a device, equipment and a storage medium for segmenting hysteromyoma of a medical image.
Background
Traditional hysteromyoma treatment methods are mostly surgical excision, but the surgical excision is large in wound.
The high-intensity focused ultrasound technology (High Intensity Focused Ultrasound, HIFU) is adopted to treat hysteromyoma, the HIFU ablation technology focuses the in-vitro low-energy ultrasound to a specific area in the body by a certain focusing technology, and the tissue is subjected to irreversible coagulation necrosis by means of the biological effect of the ultrasound, but the surrounding normal tissue is not damaged, so that the purpose of tumor ablation is achieved.
Compared with the traditional operation method, the HIFU ablation technology has the advantages of quick recovery time, repeatable treatment, uterus reservation and the like, and is widely applied worldwide. How to realize more accurate HIFU curative effect prediction before operation is important for doctors to select a treatment scheme to improve the treatment success rate. The magnetic resonance imaging (Magnetic Resonance Imaging, MRI) image has the advantages of high resolution of soft tissues and multidimensional imaging, is a key technology for diagnosis of uterine fibroids, and is an effective means for predicting and evaluating the treatment effect of HIFU operation. MRI image segmentation of uterine fibroids and uterine outlines is helpful for the formulation of preoperative schemes, intra-operative navigation and evaluation of postoperative efficacy, and is a necessary step for performing HIFU ultrasonic focusing treatment.
However, manually segmenting myomas from MRI images by a physician is a time consuming, subjective task, and the contrast between uterine myomas and other tissues in MRI images is low, making it difficult to distinguish the boundaries between them. Furthermore, the number, morphology and size of uterine fibroids vary widely from patient to patient. Thus, an accurate method for automatically segmenting uterine fibroids is critical for delineating the outline of the uterus.
More and more deep learning models are applied to medical image research, wherein the most widely used technology is based on convolutional neural networks (Convolutional Neural Network, CNN), but in practice, it is found that inaccuracy is increased by the transmitted characteristics due to indirect transmission of characteristic information, so that inaccuracy of position information and inaccuracy in segmentation of focus edges can occur, and problems of difficult recognition of small-volume tumors and poor boundary segmentation often exist, so that the reliability of segmentation prediction results is not high.
Disclosure of Invention
The invention aims to provide a method, a device, equipment and a storage medium for segmenting hysteromyoma of medical images, which can improve the identification capability of small-volume tumors, thereby improving the segmentation accuracy and reliability.
The first aspect of the invention discloses a method for segmenting hysteromyoma of medical images, which comprises the following steps:
s1, acquiring a plurality of medical images for preprocessing, and acquiring a plurality of sample images to form a sample data set;
s2, constructing a U-shaped network model; the U-shaped network model uses a Unet as a backbone network and comprises an encoder and a decoder, wherein the network structure of the encoder and the decoder is respectively divided into five layers from top to bottom, each layer of the first four layers of the encoder is connected with the corresponding layer of the decoder in a jumping manner, and the fifth layer of the encoder is connected with the fifth layer of the decoder through a serially connected hole space convolution pooling pyramid and a convolution attention module; each layer of the encoder comprises serially connected residual blocks and a first channel attention module, wherein each residual block comprises serially connected 3 convolution blocks, a batch normalization layer and an activation function are arranged behind each convolution block, the 3 convolution blocks are respectively a first convolution block, a second convolution block and a third convolution block, the first convolution block is used for adding a specified number of channels, the second convolution block is used for downsampling, the third convolution block is used for extracting features under the condition of not changing the number of channels, and the first channel attention module is used for carrying out weight assignment on a multi-channel feature map according to weight information of different channels; the cavity space convolution pooling pyramid comprises four cavity convolution layers with expansion rates of 1, 6, 12 and 18 which are stacked, and the convolution attention module comprises a second channel attention module and a space attention module which are connected in series;
s3, training a U-shaped network model by using a sample data set to obtain a hysteromyoma image segmentation model with optimal model parameters;
s4, inputting the medical image to be detected into a uterine fibroid image segmentation model to obtain a tumor area corresponding to the uterine fibroid.
In some embodiments, inputting a medical image to be measured into a uterine fibroid image segmentation model to obtain a tumor region corresponding to a uterine fibroid, including:
inputting a medical image to be detected into a uterine fibroid image segmentation model, sequentially increasing channels of the medical image to be detected from 1 to 64, 128, 256, 512 and 1024 through residual blocks of each layer of an encoder, and carrying out weight assignment on a multi-channel feature map through a first channel attention module of each layer of the encoder according to weight information of different channels to obtain a coding feature map;
when the number of channels reaches 1024, sampling the coding feature map by four cavity convolution layers at different sampling rates to obtain multi-scale features, and fusing the multi-scale features to obtain a final feature extraction result;
deducing an attention map on a channel and spatially through the second channel attention module and the spatial attention module respectively, multiplying the attention map with a feature extraction result after multi-scale feature fusion, and obtaining an adaptive feature map;
and decoding the self-adaptive feature map layer by layer through the decoder until the number of channels is reduced to 64 by a first layer of the decoder, mapping the result into probability by a 1X 1 convolution operator followed by a sigmoid activation function layer, obtaining a final segmentation result, and marking the segmentation result as a tumor area corresponding to hysteromyoma in the medical image to be detected.
In some embodiments, each layer of the decoder is provided with a double convolution module; the layer-by-layer decoding of the adaptive feature map by the decoder includes:
and fusing the input self-adaptive feature map with the features of the corresponding layers of the jump-connected encoder through the double convolution module of each layer of the encoder, and then executing up-sampling operation to restore the image size.
In some embodiments, the first convolution block increases a specified number of channels by performing a 1 x 1 convolution operation with a specified number of convolution kernels; the second convolution block performs a 3×3 convolution operation with a step size of 2 to downsample the feature map resolution; the third convolution block performs feature extraction by performing a 1 x 1 convolution operation.
In some embodiments, acquiring a number of medical images for preprocessing includes:
and carrying out uniform size operation on a plurality of medical images, adjusting the medical images to 256×256 pixels, and saving the medical images as sample images in png format.
In some embodiments, the method further comprises: and dividing the sample data set into a training set, a testing set and a verification set according to a specified proportion.
In some embodiments, the specified ratio is 8:1:1.
A second aspect of the present invention discloses a uterine fibroid segmentation device of a medical image, comprising:
the preprocessing unit is used for acquiring a plurality of medical images for preprocessing to obtain a plurality of sample images to form a sample data set;
the building unit is used for building a U-shaped network model; the U-shaped network model uses a Unet as a backbone network and comprises an encoder and a decoder, wherein the network structure of the encoder and the decoder is respectively divided into five layers from top to bottom, each layer of the first four layers of the encoder is connected with the corresponding layer of the decoder in a jumping manner, and the fifth layer of the encoder is connected with the fifth layer of the decoder through a serially connected hole space convolution pooling pyramid and a convolution attention module; each layer of the encoder comprises serially connected residual blocks and a first channel attention module, wherein each residual block comprises serially connected 3 convolution blocks, a batch normalization layer and an activation function are arranged behind each convolution block, the 3 convolution blocks are respectively a first convolution block, a second convolution block and a third convolution block, the first convolution block is used for adding a specified number of channels, the second convolution block is used for downsampling, the third convolution block is used for extracting features under the condition of not changing the number of channels, and the first channel attention module is used for carrying out weight assignment on a multi-channel feature map according to weight information of different channels; the cavity space convolution pooling pyramid comprises four cavity convolution layers with expansion rates of 1, 6, 12 and 18 which are stacked, and the convolution attention module comprises a second channel attention module and a space attention module which are connected in series;
the training unit is used for training the U-shaped network model by using the sample data set to obtain a hysteromyoma image segmentation model with optimal model parameters;
the prediction unit is used for inputting the medical image to be detected into the uterine fibroid image segmentation model to obtain a tumor area corresponding to the uterine fibroid.
A third aspect of the invention discloses an electronic device comprising a memory storing executable program code and a processor coupled to the memory; the processor invokes the executable program code stored in the memory for performing the uterine fibroid segmentation method of the medical image disclosed in the first aspect.
A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the uterine fibroid segmentation method of the medical image disclosed in the first aspect.
The invention has the beneficial effects that the U-shaped network combining the cavity convolution and the attention mechanism is provided, unimportant characteristics are restrained by applying the channel attention mechanism between each layer of the encoder, and the cavity space convolution pooling pyramid and the convolution attention module are combined at the bottom of the encoder, so that the recognition capability of a model on small-volume tumors can be improved, and the segmentation accuracy and reliability are further improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles and effects of the invention.
Unless specifically stated or otherwise defined, the same reference numerals in different drawings denote the same or similar technical features, and different reference numerals may be used for the same or similar technical features.
FIG. 1 is a flow chart of a method for segmenting uterine fibroids in a medical image in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network structure of a U-shaped network model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a portion of an encoder of a U-network model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a portion of an encoder of a U-network model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a structure of a combination of a pyramid and a convolution attention module of a cavity space convolution pooling disclosed in an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a medical image hysteromyoma segmentation device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Reference numerals illustrate:
601. a preprocessing unit; 602. a construction unit; 603. a training unit; 604. a prediction unit; 701. a memory; 702. a processor.
Detailed Description
In order that the invention may be readily understood, a more particular description of specific embodiments thereof will be rendered by reference to specific embodiments that are illustrated in the appended drawings.
Unless defined otherwise or otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. In the context of a realistic scenario in connection with the technical solution of the invention, all technical and scientific terms used herein may also have meanings corresponding to the purpose of the technical solution of the invention. The terms "first and second …" are used herein merely for distinguishing between names and not for describing a particular number or order. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
It will be understood that when an element is referred to as being "fixed" to another element, it can be directly fixed to the other element or intervening elements may also be present; when an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present; when an element is referred to as being "mounted to" another element, it can be directly mounted to the other element or intervening elements may also be present. When an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present.
As used herein, unless specifically stated or otherwise defined, "the" means that the feature or technical content mentioned or described before in the corresponding position may be the same or similar to the feature or technical content mentioned. Furthermore, the terms "comprising," "including," and "having," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a method for segmenting hysteromyoma of medical images, which can be realized by computer programming. The execution subject of the method may be an electronic device such as a computer, a notebook computer, a tablet computer, or a hysteromyoma segmentation device of a medical image embedded in the electronic device, which is not limited in the present invention.
As shown in fig. 1, the uterine fibroid segmentation method of the medical image includes the following steps S1 to S4:
s1, acquiring a plurality of medical images for preprocessing, and acquiring a plurality of sample images to form a sample data set.
In the embodiment of the invention, the MRI images of 70 uterine fibroid patients are acquired, wherein the MRI images are specifically 12-weighted or T2-weighted medical images, for example 1582T 2-weighted MRI images are acquired and marked as medical images to be preprocessed.
Wherein the pretreatment process comprises the following steps: the medical image is subjected to a uniform size operation, adjusted to 256×256 pixels, and then restored to a sample image in png format. And simultaneously anonymizing the re-saved sample image in order to protect sensitive information of the patient. The sample data set is divided into a training set, a test set and a verification set according to a specified ratio, for example, the specified ratio is set to 8:1:1, then 8/10 number of sample images are determined as the training set, 1/10 number of sample images are determined as the test set, and 1/10 number of sample images are determined as the verification set. Further, in order to enlarge the sample set, the sample images in the training set and the verification set can be randomly transformed through operations such as rotation, flipping, clipping, contrast adjustment, saturation adjustment and the like, so as to derive more sample images.
S2, constructing a U-shaped network model.
As shown in fig. 2, the U-network model uses the Unet as a backbone network, including an encoder that enlarges the receptive field as a downsampling path while reducing the computational cost, and a decoder that restores the lost resolution in the downsampling path as an upsampling path. The whole network structure is divided into five layers from top to bottom, namely, the network structure of the encoder and the decoder is respectively divided into five layers from top to bottom, each layer of the first four layers of the encoder is connected with the corresponding layer of the decoder in a jumping manner, and the fifth layer of the encoder is connected with the fifth layer of the decoder through a serially connected cavity space convolution pooling pyramid (Atrous Spatial Pyramid Pooling, ASPP) and a convolution attention module (Convolutional Block Attention Module, CBAM).
In the present invention, the residual connection combined with the channel attention mechanism is used to guarantee feature extraction capability from layer to layer of the encoder, in addition to the maximum pooling that is along with the traditional 2 x 2. As shown in fig. 3, each layer of the encoder includes a serial residual block and a first channel attention (SE) module, the residual block includes 3 convolution blocks in serial connection, a batch normalization (Batch Normalization, BN) layer and a Relu activation function are disposed after each convolution block, the 3 convolution blocks are respectively a first convolution block, a second convolution block and a third convolution block, the first convolution block is used for adding a specified number of channels, and a 1×1 convolution operation is performed with a specified number of convolution kernels to obtain a feature map of the multiple channels; for example, the first convolution block of the residual block in the first layer of the encoder is used to change the channel from 1 to 64, thus comprising 64 convolution kernels. Similarly, the first convolution block of the residual block in the other layer converts the channel to 128, 256, 512, and finally 1024 using convolution operations, respectively; the second convolution block is used for downsampling, and specifically performs a 3×3 convolution operation with a step size of 2 to downsample the feature map resolution, so as to save information and reduce calculation cost. The third convolution block is used for extracting features without changing the number of channels, specifically performing a 1×1 convolution operation, and performing feature extraction.
And the first channel attention module is used for carrying out weight assignment on the multi-channel feature map according to the weight information of different channels so as to obtain a more accurate coding feature map. Less reliable features can be suppressed by introducing residual modules into the encoder section to improve the ability of the model to extract features, while adding a first channel attention module to weight the different channels after the residual blocks of each layer.
As shown in fig. 4, the hole space convolution pooling pyramid ASPP includes four hole convolution layers with expansion rates (also called sampling rates) of 1, 6, 12 and 18 respectively, the four hole convolution layers with different sampling rates sample the coding feature map input by the first channel attention module of the last layer of the encoder at different sampling rates, and then add the sampled results together to expand the channel number, that is, fuse the obtained multi-scale features to obtain the final feature extraction result. The convolution attention module CBAM is arranged behind the ASPP, the CBAM comprises a second channel attention module and a space attention module which are connected in series, the characteristics of the multiple channels after fusion are input into the CBAM, attention force diagrams are inferred on the channels and in space through the second channel attention module and the space attention module respectively, then the attention force diagrams are multiplied with the input multi-scale characteristics to carry out self-adaptive characteristic optimization, and a self-adaptive characteristic diagram is obtained, so that the focus position of a model is more accurately positioned, and the model segmentation accuracy is improved.
Further, the decoder performs an up-sampling sensing/up-sampling operation on the adaptive feature map having the smallest size, thereby performing decoding. Features in the downsampled path are connected to features in the upsampled path by a jump connection to provide added information without downsampling information abstraction. Each layer of the decoder is provided with a double convolution module, firstly, the double convolution module is fused with the features of the encoder which are connected in a jumping manner, then the image size is restored by using upsampling, finally, the number of channels is reduced to 64, and the result is mapped into probability through a 1 multiplied by 1 convolution operator followed by a sigmoid activation function layer, so that the final segmentation result is obtained.
As an alternative embodiment, an intermediate feature map F ε R is given C×H×W As input, where R is a matrix, C is the number of channels, H and W represent the height and width of the image. First, spatial information of feature maps is aggregated using an average pooling and a maximum pooling operation to generate two different spatial context descriptors: f (F) c avg And F c max . To generate channel attention map M c (f) The feature map is compressed here by a shared network of Multi-Layer Perceptron (MLP) and hidden layers, with a compression ratio r. The attention calculation formulas of the first channel attention module and the second channel attention module are shown as the following formula (1):
M c (f)＝σ(MLP(AvgPool/(f))+MLP(MaxPool/(f)))
＝σ(W 1 (W 0 (F c avg ))+W 1 (W 0 (F c max ))) (1)
where σ is a sigmoid activation function, W 0 ∈R C/r × C ，W 1 ∈R C × C/r ；W 0 、W 1 Is the weight of the multi-layer perceptron, shares the input and W 0 Is activated by a RELU.
As an alternative embodiment, two pooling operations are used to aggregate the channel information of the functional map, generating two 2-dimensional maps: f (F) s avg ∈R 1×HxW And F s max ∈R 1×HxW Mean pooling and maximum pooling are indicated. The attention calculation formula of the spatial attention module is shown in the following formula (2):
M s (f)＝σ(f 7×7 (AvgPool/(f)；MaxPool/(F)])))
＝σ(f 7×7 (F s avg ；F s max ])) (2)
where σ is a sigmoid activation function, f 7×7 Is a 7 x 7 convolution.
And S3, training a U-shaped network model by using the sample data set to obtain a hysteromyoma image segmentation model with optimal model parameters.
The loss function is used for evaluating the degree of inconsistency between the predicted value and the actual value of the model, and the loss function is reduced in the training process of the network model, which means that the closer the predicted value is to the actual value, the better the robustness of the network model is. A cross entropy loss function (BinaryCross Entropy) is used in the training of the network model of an embodiment of the invention, which calculates a loss value, i.e. loss value, using a bi-classification model with an output dimension of 1. BCEWithLogitsLoss is defined as:
BCE_Loss＝{l 1 ,…,l N },l n ＝-[y n ·logσ(x n )+(1-y n )·log(1-σ(x n ))]
(3)
where σ is a sigmoid activation function, x can be mapped to the interval of (0, 1):
the optimizer employed in the embodiments of the present invention is a random gradient descent (SGD) optimizer with an initial momentum of 0.9. Early stop method early stop is used in the process of training the network model to prevent the network model from being over fitted. And (3) saving the parameter of the trained uterine tumor image segmentation model with highest accuracy in the verification set as the optimal model parameter.
Wherein, different quantization indexes are used to comprehensively evaluate the segmentation performance of the method and other existing segmentation methods. These metrics include the Dice similarity coefficient, the jaccard coefficient (Jaccard Coefficient, JC), the hausdorff distance (Hausdorff Distance, HD), the average surface distance (average surface distance, ASD), and the recall (recall).
Wherein the Dice coefficient is one of the most commonly used medical segmentation evaluation indexes, as shown in the following formula (5):
wherein A and B represent gold standard and predicted image, respectively.
As another expression method of the Dice coefficient, the following formula (6) shows:
where TP represents the positive value when the myoma pixels in the gold standard are correctly classified in the predicted image, FN misclassifying the myoma pixels to the non-myoma pixels in the predicted image, FP means that the non-myoma pixels in the gold standard are misclassifying as myoma pixels in the predicted image.
Wherein the jaccard coefficient JC is used to compare the differences and similarities between sample sets, the higher the jaccard coefficient the more similar the two samples are. The calculation mode of the Jacquard coefficient is shown as the following formula (7):
where HD is the distance between two subsets in metric space, calculated as shown in equation (8):
H(A,B)＝max{h(A,B),h(B,A)}
(8)
ASD represents the average surface distance between the segmentation result and the gold standard, calculated as shown in the following equation (9):
from the perspective of the real results, recall describes how many real positive examples in the test set were picked by the classifier, i.e., how many real positive examples were recalled by the classifier. The calculation method of Recall is shown in the following formula (10):
the saved model parameters are used for automatic prediction of the tumor area on the uterine tumor test set, and the segmentation result of other existing models on the test set is shown in fig. 5, and the quantitative results are compared with the following table 1.
Table 1 comparison of the quantitative results of the segmentation results for the test set
S4, inputting the medical image to be detected into a uterine fibroid image segmentation model to obtain a tumor area corresponding to the uterine fibroid.
In practical application, an unknown medical image to be detected is input into a uterine fibroid image segmentation model, and a tumor area corresponding to uterine fibroid can be obtained predictively. Specifically, step S4 may include steps S41 to S44, which are not shown below:
s41, inputting a medical image to be detected into a uterine fibroid image segmentation model, sequentially increasing channels of the medical image to be detected from 1 to 64, 128, 256, 512 and 1024 through residual blocks of each layer of an encoder, and carrying out weight assignment on the multi-channel feature images according to weight information of different channels through a first channel attention module of each layer of the encoder to obtain a coding feature image;
s42, when the number of channels reaches 1024, sampling the coding feature map by four cavity convolution layers at different sampling rates to obtain multi-scale features, and fusing the multi-scale features to obtain a final feature extraction result;
s43, deducing an attention map on a channel and in space respectively through the second channel attention module and the space attention module, multiplying the attention map by a feature extraction result after multi-scale feature fusion, and obtaining an adaptive feature map;
s44, decoding the self-adaptive feature map layer by layer through the decoder until the number of channels is reduced to 64 by a first layer of the decoder, mapping the result into probability by a 1X 1 convolution operator followed by a sigmoid activation function layer, obtaining a final segmentation result, and marking the segmentation result as a tumor area corresponding to hysteromyoma in the medical image to be detected.
Further, performing layer-by-layer decoding on the adaptive feature map by the decoder may include:
and fusing the input self-adaptive feature map with the features of the corresponding layers of the jump-connected encoder through the double convolution module of each layer of the encoder, and then executing up-sampling operation to restore the image size.
In summary, according to the characteristics of variable shapes, large size difference, unknown quantity and low contrast between adjacent organs among hysteromyoma individuals, the embodiment of the invention combines a spatial attention mechanism and a channel attention mechanism besides the capability of improving the characteristic extraction of a model by using residual connection in an encoder, and connects CBAM and ASPP at the lowest layer of the encoder, thereby improving the capability of extracting the characteristic of the model, further improving the identification capability of small-volume tumors and further improving the segmentation accuracy and reliability.
As shown in fig. 6, an embodiment of the present invention discloses a device for dividing hysteromyoma of medical image, comprising a preprocessing unit 601, a construction unit 602, a training unit 603, and a prediction unit 604, wherein,
a preprocessing unit 601, configured to acquire a plurality of medical images for preprocessing, and obtain a plurality of sample images to form a sample data set;
a construction unit 602, configured to construct a U-type network model; the U-shaped network model uses a Unet as a backbone network and comprises an encoder and a decoder, wherein the network structure of the encoder and the decoder is respectively divided into five layers from top to bottom, each layer of the first four layers of the encoder is connected with the corresponding layer of the decoder in a jumping manner, and the fifth layer of the encoder is connected with the fifth layer of the decoder through a serially connected hole space convolution pooling pyramid and a convolution attention module; each layer of the encoder comprises serially connected residual blocks and a first channel attention module, wherein each residual block comprises serially connected 3 convolution blocks, a batch normalization layer and an activation function are arranged behind each convolution block, the 3 convolution blocks are respectively a first convolution block, a second convolution block and a third convolution block, the first convolution block is used for adding a specified number of channels, the second convolution block is used for downsampling, the third convolution block is used for extracting features under the condition of not changing the number of channels, and the first channel attention module is used for carrying out weight assignment on a multi-channel feature map according to weight information of different channels; the cavity space convolution pooling pyramid comprises four cavity convolution layers with expansion rates of 1, 6, 12 and 18 which are stacked, and the convolution attention module comprises a second channel attention module and a space attention module which are connected in series;
a training unit 603, configured to train the U-shaped network model by using the sample data set, and obtain a uterine fibroid image segmentation model with optimal model parameters;
and the prediction unit 604 is used for inputting the medical image to be detected into the uterine fibroid image segmentation model to obtain a tumor area corresponding to the uterine fibroid.
As shown in fig. 7, an embodiment of the present invention discloses an electronic device including a memory 701 storing executable program code and a processor 702 coupled to the memory 701;
wherein the processor 702 invokes executable program code stored in the memory 701 to perform the uterine fibroid segmentation method of the medical image described in the above embodiments.
The embodiments of the present invention also disclose a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the uterine fibroid segmentation method of the medical image described in the above embodiments.
The foregoing embodiments are provided for the purpose of exemplary reproduction and deduction of the technical solution of the present invention, and are used for fully describing the technical solution, the purpose and the effects of the present invention, and are used for enabling the public to understand the disclosure of the present invention more thoroughly and comprehensively, and are not used for limiting the protection scope of the present invention.
The above examples are also not an exhaustive list based on the invention, and there may be a number of other embodiments not listed. Any substitutions and modifications made without departing from the spirit of the invention are within the scope of the invention.
Claims (10)
1. A method for segmenting uterine fibroid in a medical image, comprising:
s1, acquiring a plurality of medical images for preprocessing, and acquiring a plurality of sample images to form a sample data set;
s2, constructing a U-shaped network model; the U-shaped network model uses a Unet as a backbone network and comprises an encoder and a decoder, wherein the network structure of the encoder and the decoder is respectively divided into five layers from top to bottom, each layer of the first four layers of the encoder is connected with the corresponding layer of the decoder in a jumping manner, and the fifth layer of the encoder is connected with the fifth layer of the decoder through a serially connected hole space convolution pooling pyramid and a convolution attention module; each layer of the encoder comprises serially connected residual blocks and a first channel attention module, wherein each residual block comprises serially connected 3 convolution blocks, a batch normalization layer and an activation function are arranged behind each convolution block, the 3 convolution blocks are respectively a first convolution block, a second convolution block and a third convolution block, the first convolution block is used for adding a specified number of channels, the second convolution block is used for downsampling, the third convolution block is used for extracting features under the condition of not changing the number of channels, and the first channel attention module is used for carrying out weight assignment on a multi-channel feature map according to weight information of different channels; the cavity space convolution pooling pyramid comprises four cavity convolution layers with expansion rates of 1, 6, 12 and 18 which are stacked, and the convolution attention module comprises a second channel attention module and a space attention module which are connected in series;
s3, training a U-shaped network model by using a sample data set to obtain a hysteromyoma image segmentation model with optimal model parameters;
s4, inputting the medical image to be detected into a uterine fibroid image segmentation model to obtain a tumor area corresponding to the uterine fibroid.
2. The method for segmenting uterine fibroid in a medical image according to claim 1, wherein inputting the medical image to be detected into the uterine fibroid image segmentation model to obtain a tumor region corresponding to the uterine fibroid comprises:
inputting a medical image to be detected into a uterine fibroid image segmentation model, sequentially increasing channels of the medical image to be detected from 1 to 64, 128, 256, 512 and 1024 through residual blocks of each layer of an encoder, and carrying out weight assignment on a multi-channel feature map through a first channel attention module of each layer of the encoder according to weight information of different channels to obtain a coding feature map;
when the number of channels reaches 1024, sampling the coding feature map by four cavity convolution layers at different sampling rates to obtain multi-scale features, and fusing the multi-scale features to obtain a final feature extraction result;
deducing an attention map on a channel and spatially through the second channel attention module and the spatial attention module respectively, multiplying the attention map with a feature extraction result after multi-scale feature fusion, and obtaining an adaptive feature map;
and decoding the self-adaptive feature map layer by layer through the decoder until the number of channels is reduced to 64 by a first layer of the decoder, mapping the result into probability by a 1X 1 convolution operator followed by a sigmoid activation function layer, obtaining a final segmentation result, and marking the segmentation result as a tumor area corresponding to hysteromyoma in the medical image to be detected.
3. A method of segmentation of a uterine fibroid of a medical image according to claim 2, characterized in that each layer of the decoder is provided with a double convolution module; the layer-by-layer decoding of the adaptive feature map by the decoder includes:
and fusing the input self-adaptive feature map with the features of the corresponding layers of the jump-connected encoder through the double convolution module of each layer of the encoder, and then executing up-sampling operation to restore the image size.
4. A method for segmenting a uterine fibroid in a medical image according to claim 1,
the first convolution block increases a specified number of channels by performing a 1 x 1 convolution operation with a specified number of convolution kernels;
the second convolution block performs a 3×3 convolution operation with a step size of 2 to downsample the feature map resolution;
the third convolution block performs feature extraction by performing a 1 x 1 convolution operation.
5. A method of segmentation of a uterine fibroid in a medical image according to any of claims 1-4, characterized in that acquisition of several medical images for preprocessing comprises:
and carrying out uniform size operation on a plurality of medical images, adjusting the medical images to 256×256 pixels, and saving the medical images as sample images in png format.
6. A method of segmentation of a uterine fibroid of a medical image according to any of claims 1-4, characterized in that the method further comprises:
and dividing the sample data set into a training set, a testing set and a verification set according to a specified proportion.
7. The method for segmenting hysteromyoma in medical images according to claim 6, characterized in that the prescribed ratio is 8:1:1.
8. A uterine fibroid segmentation device of a medical image, comprising:
the preprocessing unit is used for acquiring a plurality of medical images for preprocessing to obtain a plurality of sample images to form a sample data set;
the building unit is used for building a U-shaped network model; the U-shaped network model uses a Unet as a backbone network and comprises an encoder and a decoder, wherein the network structure of the encoder and the decoder is respectively divided into five layers from top to bottom, each layer of the first four layers of the encoder is connected with the corresponding layer of the decoder in a jumping manner, and the fifth layer of the encoder is connected with the fifth layer of the decoder through a serially connected hole space convolution pooling pyramid and a convolution attention module; each layer of the encoder comprises serially connected residual blocks and a first channel attention module, wherein each residual block comprises serially connected 3 convolution blocks, a batch normalization layer and an activation function are arranged behind each convolution block, the 3 convolution blocks are respectively a first convolution block, a second convolution block and a third convolution block, the first convolution block is used for adding a specified number of channels, the second convolution block is used for downsampling, the third convolution block is used for extracting features under the condition of not changing the number of channels, and the first channel attention module is used for carrying out weight assignment on a multi-channel feature map according to weight information of different channels; the cavity space convolution pooling pyramid comprises four cavity convolution layers with expansion rates of 1, 6, 12 and 18 which are stacked, and the convolution attention module comprises a second channel attention module and a space attention module which are connected in series;
the training unit is used for training the U-shaped network model by using the sample data set to obtain a hysteromyoma image segmentation model with optimal model parameters;
the prediction unit is used for inputting the medical image to be detected into the uterine fibroid image segmentation model to obtain a tumor area corresponding to the uterine fibroid.
9. An electronic device comprising a memory storing executable program code and a processor coupled to the memory; the processor invokes the executable program code stored in the memory for performing the uterine fibroid segmentation method of the medical image of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, wherein the computer program causes a computer to execute the uterine fibroid segmentation method of a medical image according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311336190.XA CN117253045A (en) | 2023-10-16 | 2023-10-16 | Hysteromyoma segmentation method and device of medical image, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311336190.XA CN117253045A (en) | 2023-10-16 | 2023-10-16 | Hysteromyoma segmentation method and device of medical image, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117253045A true CN117253045A (en) | 2023-12-19 |
Family
ID=89136768
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311336190.XA Pending CN117253045A (en) | 2023-10-16 | 2023-10-16 | Hysteromyoma segmentation method and device of medical image, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117253045A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117831122A (en) * | 2023-12-20 | 2024-04-05 | 慧之安信息技术股份有限公司 | Underground vehicle-booking method and system based on gesture recognition |
-
2023
- 2023-10-16 CN CN202311336190.XA patent/CN117253045A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117831122A (en) * | 2023-12-20 | 2024-04-05 | 慧之安信息技术股份有限公司 | Underground vehicle-booking method and system based on gesture recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112070119B (en) | Ultrasonic section image quality control method, device and computer equipment | |
| KR102125127B1 (en) | Method of brain disorder diagnosis via deep learning | |
| US11633169B2 (en) | Apparatus for AI-based automatic ultrasound diagnosis of liver steatosis and remote medical diagnosis method using the same | |
| EP3046478B1 (en) | Image analysis techniques for diagnosing diseases | |
| Zhang et al. | Intelligent scanning: Automated standard plane selection and biometric measurement of early gestational sac in routine ultrasound examination | |
| CN109614991A (en) | A kind of segmentation and classification method of the multiple dimensioned dilatancy cardiac muscle based on Attention | |
| CN113850753B (en) | Medical image information computing method, device, edge computing equipment and storage medium | |
| CN107220966A (en) | A kind of Histopathologic Grade of Cerebral Gliomas Forecasting Methodology based on image group | |
| CN117253045A (en) | Hysteromyoma segmentation method and device of medical image, equipment and storage medium | |
| JP2020010805A (en) | Specification device, program, specification method, information processing device, and specifier | |
| CN114565613B (en) | Pancreas postoperative diabetes prediction system based on there is study of supervision degree of depth subspace | |
| CN114119515A (en) | Brain tumor detection method based on attention mechanism and MRI multi-mode fusion | |
| KR20220144687A (en) | Dual attention multiple instance learning method | |
| CN112465771A (en) | Method and device for analyzing spine nuclear magnetic resonance image and computer equipment | |
| CN113707278B (en) | Brain CT medical report generation method based on spatial coding | |
| CN115249228A (en) | Chest X-ray image identification method, chest X-ray image identification device, computer equipment and storage medium | |
| CN111899848B (en) | Image recognition method and device | |
| CN117350979A (en) | Arbitrary focus segmentation and tracking system based on medical ultrasonic image | |
| KR102391934B1 (en) | System and method for diagnosis cancer risk of thyroid nodule based on artificial intelligence | |
| Sha et al. | A robust segmentation method based on improved U-Net | |
| CN114937044A (en) | Lightweight image segmentation method and device and storage medium | |
| CN115311317A (en) | Laparoscope image segmentation method and system based on ScaleFormer algorithm | |
| CN114581459A (en) | Improved 3D U-Net model-based segmentation method for image region of interest of preschool child lung | |
| KR102258070B1 (en) | Method for evaluating foot type and device evaluating foot type using the same | |
| Lasala et al. | Exploiting class activation mappings as prior to generate fetal brain ultrasound images with GANs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |