CN117764948A - Liver tumor segmentation method based on mixed attention and multi-scale supervision - Google Patents

Abstract

The invention relates to the field of medical image processing, in particular to a liver tumor segmentation method based on mixed attention and multi-scale supervision, which comprises the following steps: preprocessing the acquired CT image of the liver tumor to be segmented; according to a MAS-ConvUNeXt liver tumor segmentation model comprising a mixed attention module and a multi-scale supervision module, carrying out liver tumor segmentation on a CT image of the liver tumor to be segmented, wherein the training method of the segmentation model comprises the following steps: and acquiring an image data set formed by the CT images of the liver tumor, preprocessing, and training the constructed segmentation model according to the preprocessed image data set. The MAS-ConvUNeXt liver tumor segmentation model provided by the invention can reduce the number of parameters while increasing the receptive field of the neural network, and the segmentation quality is enhanced by the mixed attention module, so that the liver tumor can be segmented efficiently and rapidly, and the segmentation accuracy of liver tumor images is improved.

Description

A liver tumor segmentation method based on hybrid attention and multi-scale supervision

Technical field

The invention relates to the field of medical image processing, and specifically relates to a liver tumor segmentation method based on hybrid attention and multi-scale supervision.

Background technique

Liver cancer is one of the leading causes of death in the world. Using CT (Computed Tomography) can help doctors detect liver cancer early. Accurate liver tumor segmentation results can facilitate doctors to observe liver tumor conditions, and can assist doctors in formulating treatment plans, surgical planning and monitoring treatment.

Due to the complexity and volume of medical imaging data, traditional manual segmentation methods become time-consuming and error-prone. Automated liver tumor segmentation algorithms can greatly improve efficiency and reduce doctors' workload. Using liver tumor segmentation algorithms, doctors can analyze medical images more quickly and accurately, improving the sensitivity and specificity of diagnosis.

In recent years, deep learning segmentation networks, especially architectures such as UNet, have made significant progress in liver tumor segmentation tasks. The deep learning model can automatically learn features and extract information in a hierarchical manner to better adapt to the complex and changeable liver structure and tumor morphology. However, traditional UNet will obtain a lot of redundant information after simple image splicing, and will lose more semantic features during the upsampling process. The segmentation accuracy for liver tumors is often low.

Summary of the invention

In order to solve the technical problem of low liver tumor image segmentation accuracy, the present invention proposes a liver tumor segmentation method based on hybrid attention and multi-scale supervision.

The present invention provides a liver tumor segmentation method based on hybrid attention and multi-scale supervision, which method includes:

Obtain CT images of liver tumors to be segmented, and preprocess the CT images of liver tumors to be segmented;

According to the pre-trained MAS-ConvUNeXt liver tumor segmentation model, liver tumor segmentation is performed on the liver tumor CT image to be segmented. The training method of the MAS-ConvUNeXt liver tumor segmentation model includes: obtaining an image data set composed of liver tumor CT images, Preprocess the image data set and construct the MAS-ConvUNeXt liver tumor segmentation model. Based on the preprocessed image data set, train the constructed MAS-ConvUNeXt liver tumor segmentation model to obtain the trained MAS-ConvUNeXt liver tumor segmentation. Model, MAS-ConvUNeXt liver tumor segmentation model includes: hybrid attention module and multi-scale supervision module.

Optionally, the acquisition of an image data set composed of liver tumor CT images includes:

The 3D liver tumor CT image is processed into a 2D image containing the liver and liver tumors as a liver tumor CT image, and all the obtained liver tumor CT images are formed into an image data set.

Optionally, the preprocessing includes: image enhancement.

Optionally, the construction of the MAS-ConvUNeXt liver tumor segmentation model includes:

Construct a ConvUNeXt model, where the ConvUNeXt model includes: encoder, decoder and attention gating module;

Based on the ConvUNeXt model, its attention gating module is improved to a hybrid attention module, and a multi-scale supervision module is added;

The finally updated ConvUNeXt model is used as the constructed pre-training MAS-ConvUNeXt liver tumor segmentation model.

Optionally, the encoder and decoder are based on the encoder and decoder of UNet, using the ConvUNeXt convolution block to replace the convolution block of UNet; the stage ratio of the encoder is 1:1:3:1, the convolutional layer is used to replace the pooling layer in the downsampling step; the attention gating module is located in the skip connection part.

Optionally, the hybrid attention module linearly transforms the result obtained after upsampling the output feature map x1 of the lower ConvUNeXt convolution block, and divides the resulting feature map into 1:2 in the channel dimension, and divides the number of channels into The feature map with a ratio of 1 is input into the spatial attention module, the feature map with a channel number ratio of 2 is input into the channel attention module, and the attention weights α and β obtained by the two attention modules are convolved with the corresponding layer ConvUNeXt in the encoder The output feature map x2 of the block is multiplied and transformed by a linear layer, and then concatenated with the upsampled input feature map x1.

Optionally, the spatial attention module inputs the input feature map into the linear layer and adds it to the output feature map x2 of the corresponding layer ConvUNext convolution block in the encoder after linear transformation. The addition result uses ReLU. The activation function is activated, and then the feature map is mapped to the intermediate space through a linear layer, and finally the Sigmoid activation function is used to activate to obtain the attention weight α.

Optionally, the channel attention module performs global average pooling on the input feature map, and adds the result of global pooling to the output feature map x2 of the ConvUNext convolution block of the corresponding layer of the encoder, and the added result Use the ReLU activation function to activate, then compress and expand the feature map dimensions through two linear layers, and finally use the Sigmoid activation function to activate to obtain the attention weight β.

Optionally, the multi-scale supervision module is to perform 1x1 convolution on the feature map after each upsampling convolution in the decoder and then upsample to the original size to obtain the segmentation map of each upsampling stage. Establish a loss function with real labels to achieve multi-scale supervision.

Optionally, the loss function during the training process of the MAS-ConvUNeXt liver tumor segmentation model is the cross-entropy loss function.

The invention has the following beneficial effects:

The present invention's liver tumor segmentation method based on hybrid attention and multi-scale supervision can better solve the problem of liver tumor segmentation. The present invention uses a hybrid attention module to segment the upsampling results by channels, and applies spatial attention and Channel attention, enhances the quality of far-hop connection splicing, filters out irrelevant features in low-level semantic feature maps, enhances the model's focus on ROI areas, and adds a multi-scale supervision module, which is added at each upsampling stage in the decoder The supervision signal helps to enhance the quality of the feature map obtained by the decoder, optimizes the update process of network model parameters, and makes the model loss function converge better, thus improving the accuracy of liver tumor image segmentation.

Description of the drawings

In order to more clearly explain the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description The drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of a liver tumor segmentation method based on hybrid attention and multi-scale supervision of the present invention;

Figure 2 is a schematic diagram of the framework of the MAS-ConvUNeXt liver tumor segmentation model of the present invention;

Figure 3 is a schematic structural diagram of the ConvUNext convolution block of the present invention;

Figure 4 is a schematic structural diagram of the hybrid attention module of the present invention;

Figure 5 is a schematic structural diagram of the multi-scale supervision module of the present invention;

Figure 6 is another flow chart of the present invention.

Detailed ways

In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended inventive purpose, the following is a detailed description of the specific implementation, structure, characteristics and effects of the technical solution proposed according to the present invention in conjunction with the accompanying drawings and preferred embodiments. described as follows. In the following description, different terms "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, the specific features, structures, or characteristics of one or more embodiments may be combined in any suitable form.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which the invention belongs.

The present invention provides a liver tumor segmentation method based on hybrid attention and multi-scale supervision. The method includes the following steps:

Obtain CT images of liver tumors to be segmented, and preprocess the CT images of liver tumors to be segmented;

According to the pre-trained MAS-ConvUNeXt liver tumor segmentation model, liver tumor segmentation is performed on the liver tumor CT image to be segmented. The training method of the MAS-ConvUNeXt liver tumor segmentation model includes: obtaining an image data set composed of liver tumor CT images, Preprocess the image data set and construct the MAS-ConvUNeXt liver tumor segmentation model. Based on the preprocessed image data set, train the constructed MAS-ConvUNeXt liver tumor segmentation model to obtain the trained MAS-ConvUNeXt liver tumor segmentation. Model, MAS-ConvUNeXt liver tumor segmentation model includes: hybrid attention module and multi-scale supervision module.

Each of the above steps is expanded upon in detail below:

Referring to FIG. 1 , a flow chart of some embodiments of a liver tumor segmentation method based on hybrid attention and multi-scale supervision according to the present invention is shown. The liver tumor segmentation method based on hybrid attention and multi-scale supervision includes the following steps:

Step S1: Obtain the CT image of the liver tumor to be segmented, and preprocess the CT image of the liver tumor to be segmented.

In some embodiments, CT images of liver tumors can be collected through CT equipment as CT images of liver tumors to be segmented, and histogram averaging and contrast stretching are performed on the CT images of liver tumors to be segmented to realize CT images of liver tumors to be segmented. Enhancement to achieve preprocessing of CT images of liver tumors to be segmented.

The CT image of the liver tumor to be segmented may be a CT (Computed Tomography, computed tomography) image of the liver tumor to be segmented. Preprocessing includes: image enhancement.

Step S2, perform liver tumor segmentation on the CT image of the liver tumor to be segmented based on the pre-trained MAS-ConvUNeXt liver tumor segmentation model.

In some embodiments, the liver tumor CT image to be segmented can be input into the pre-trained MAS-ConvUNeXt liver tumor segmentation model. Through the pre-trained MAS-ConvUNeXt liver tumor segmentation model, the liver tumor CT image to be segmented can be realized. Liver tumor segmentation. Among them, the overall framework of the MAS-ConvUNeXt liver tumor segmentation model can be shown in Figure 2. MAS-ConvUNext can be recorded as Mixed Attention and multi-scaleSupervision ConvUNext.

Optionally, the training method of the MAS-ConvUNeXt liver tumor segmentation model includes the following steps:

The first step is to obtain an image data set consisting of liver tumor CT images.

For example, the 3D liver tumor CT image is processed into a 2D image containing the liver and liver tumors, including adjusting the window width and window level, and using the obtained 2D image as a liver tumor CT image, and combining all the obtained liver tumor CT images to form an image. data set. The liver tumor CT image may be a CT image with the liver tumor area marked. For example, liver tumor areas can be manually marked on liver tumor CT images.

The second step is to preprocess the image data set.

For example, the processed 2D image, that is, the liver tumor CT image in the image data set, can be subjected to histogram averaging and contrast stretching to enhance the liver tumor CT image, thereby enhancing the image data set. Perform data enhancement on the enhanced image data set, amplify the data set, and use the final image data set as the preprocessed image data set. The preprocessed image data set can be divided into a training set, a verification set, and a test set. Among them, data enhancement can include: random horizontal flipping, random scaling and random cropping. Data augmentation can generate new data with the purpose of expanding and diversifying the training data set, which can improve the generalization ability and robustness of the model.

The third step is to construct the MAS-ConvUNeXt liver tumor segmentation model.

Among them, the MAS-ConvUNeXt liver tumor segmentation model includes: a hybrid attention module and a multi-scale supervision module.

For example, building the MAS-ConvUNeXt liver tumor segmentation model can include the following sub-steps:

The first sub-step is to build the ConvUNeXt model.

Among them, the ConvUNeXt model includes: encoder, decoder and attention gating module. The above encoder and decoder are based on the encoder and decoder of UNet, using ConvUNeXt convolution block to replace the convolution block of UNet. The ConvUNext convolution block structure is shown in Figure 3. The stage ratio of the above encoder is 1:1:3:1, and a convolutional layer is used to replace the pooling layer in the downsampling step. The above attention gating module is located in the skip connection part.

The second sub-step is based on the ConvUNeXt model, improving its attention gating module into a hybrid attention module and adding a multi-scale supervision module.

Among them, the structure of the hybrid attention module can be shown in Figure 4. The above-mentioned hybrid attention module linearly transforms the result obtained by upsampling the output feature map x1 of the lower ConvUNeXt convolution block, and divides the resulting feature map into 1:2 in the channel dimension, in which the channel number is 1 The feature map is input into the spatial attention module, and the feature map with a channel number of 2 is input into the channel attention module. Finally, the attention weights α and β obtained by the two attention modules and the output of the corresponding layer ConvUNext convolution block in the encoder are input The feature map x2 is multiplied to obtain the output result of the hybrid attention module. After transforming the dimension through a linear layer, the result is spliced with the upsampled input feature map x1. The spliced feature map will be input to the next ConvUNeXt convolution block. Among them, the feature map x1 refers to the output feature map of the decoder convolution block in the lower layer of the hybrid attention module in the structure diagram. The feature map x2 refers to the output feature map of the encoder convolutional block in the structure graph that is at the same layer as the hybrid attention module.

The above-mentioned spatial attention module inputs the input feature map into the linear layer and adds it to the output feature map x2 of the corresponding layer ConvUNext convolution block in the encoder after linear transformation. The addition result at this time uses the ReLU activation function. Activation, then mapping the feature map to the intermediate space through a linear layer, and finally using the Sigmoid activation function to activate to obtain the attention weight α.

The above-mentioned channel attention module performs global average pooling on the input feature map and adds it to the output feature map x2 of the corresponding layer ConvUNext convolution block in the encoder after global pooling. The addition result at this time is used The ReLU activation function is activated, and then the feature map dimensions are compressed and expanded through two linear layers, and finally the Sigmoid activation function is used to activate to obtain the attention weight β.

The structure of the multi-scale supervision module can be shown in Figure 5. The above-mentioned multi-scale supervision module is to perform 1x1 convolution on the feature map after each upsampling in the decoder and then upsample to the original size to obtain the segmentation map of each upsampling stage, and establish a loss with the real label function to implement multi-scale supervision.

The third sub-step is to use the finally updated ConvUNeXt model as the constructed pre-training MAS-ConvUNeXt liver tumor segmentation model.

The fourth step is to train the constructed MAS-ConvUNeXt liver tumor segmentation model based on the preprocessed image data set to obtain the trained MAS-ConvUNeXt liver tumor segmentation model.

Among them, the loss function during the training process of the MAS-ConvUNeXt liver tumor segmentation model is the cross-entropy loss function.

For example, construct a loss function and train the MAS-ConvUNeXt liver tumor segmentation model on the training set, optimize the segmentation model parameters through the gradient descent method, and select the optimal segmentation model. The loss function is the cross-entropy loss function. The corresponding formula is:

Among them, L is the loss function value, N is the total number of pixels in the segmentation map, c is the category index, y _{i, c} is the true label of category c in pixel i, is the predicted probability of model output for category c in sample i.

Optionally, the images in the test set can be input into the MAS-ConvUNeXt liver tumor segmentation model to obtain the liver tumor segmentation results of the images in the test set.

Another flow chart of the present invention can be shown in Figure 6.

In summary, the present invention uses a hybrid attention module as the attention gating module and adds a multi-scale supervision module. This model can increase the receptive field of the neural network while reducing the number of parameters, and enhance the quality of the segmentation area through the hybrid attention module, achieving efficient and fast segmentation of liver tumors.

The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions of the foregoing embodiments. Modifications are made to the recorded technical solutions, or equivalent substitutions are made to some of the technical features; these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of each embodiment of the present invention, and shall be included in the protection of the present invention. within the range.

Claims (10)

1. A liver tumor segmentation method based on hybrid attention and multi-scale supervision, which is characterized by including the following steps: Obtain CT images of liver tumors to be segmented, and preprocess the CT images of liver tumors to be segmented; According to the pre-trained MAS-ConvUNeXt liver tumor segmentation model, liver tumor segmentation is performed on the liver tumor CT image to be segmented. The training method of the MAS-ConvUNeXt liver tumor segmentation model includes: obtaining an image data set composed of liver tumor CT images, Preprocess the image data set and construct the MAS-ConvUNeXt liver tumor segmentation model. Based on the preprocessed image data set, train the constructed MAS-ConvUNeXt liver tumor segmentation model to obtain the trained MAS-ConvUNeXt liver tumor segmentation. Model, MAS-ConvUNeXt liver tumor segmentation model includes: hybrid attention module and multi-scale supervision module. 2. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 1, characterized in that the acquisition of an image data set composed of liver tumor CT images includes: The 3D liver tumor CT image is processed into a 2D image containing the liver and liver tumors as a liver tumor CT image, and all the obtained liver tumor CT images are formed into an image data set. 3. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 1, characterized in that the preprocessing includes: image enhancement. 4. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 1, characterized in that the construction of the MAS-ConvUNeXt liver tumor segmentation model includes: Construct a ConvUNeXt model, where the ConvUNeXt model includes: encoder, decoder and attention gating module; Based on the ConvUNeXt model, its attention gating module is improved to a hybrid attention module, and a multi-scale supervision module is added; The finally updated ConvUNeXt model is used as the constructed pre-training MAS-ConvUNeXt liver tumor segmentation model. 5. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 4, characterized in that the encoder and decoder are based on the encoder and decoder of UNet, using ConvUNeXt The encoder and decoder obtained by replacing the convolution block of UNet with the convolution block; the stage ratio of the encoder is 1:1:3:1, and the convolution layer is used to replace the pooling layer in the downsampling step; the attention gate The control module is located in the jump connection section. 6. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 4, characterized in that the hybrid attention module upsamples the output feature map x1 of the lower layer ConvUNeXt convolution block to obtain The result is linearly transformed, and the resulting feature map is divided into 1:2 in the channel dimension. The feature map with a channel number of 1 is input into the spatial attention module, and the feature map with a channel number of 2 is input into the channel attention module. The force module multiplies the attention weights α and β obtained by the two attention modules and the output feature map x2 of the corresponding layer ConvUNext convolution block in the encoder, and transforms the dimension through a linear layer, and then combines it with the upsampled input feature Picture x1 splicing. 7. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 6, characterized in that the spatial attention module inputs the input feature map into the linear layer and combines it with the encoder. The output feature map x2 of the corresponding layer ConvUNext convolution block is added after linear transformation. The addition result is activated using the ReLU activation function, and then the feature map is mapped to the intermediate space through a linear layer. Finally, the Sigmoid activation function is used to activate, and we get Attention weight α. 8. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 6, characterized in that the channel attention module performs global average pooling on the input feature map and combines it with the encoding The output feature map x2 of the corresponding layer ConvUNext convolution block in the device is added after global pooling. The addition result is activated using the ReLU activation function, and then the feature map dimensions are compressed and expanded through two linear layers, and finally activated using Sigmoid The function is activated to obtain the attention weight β. 9. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 4, characterized in that the multi-scale supervision module is after the convolution after each upsampling in the decoder, Perform 1x1 convolution on the feature map and then upsample it to the original size to obtain the segmentation map of each upsampling stage, and establish a loss function with the real label to achieve multi-scale supervision. 10. A liver tumor segmentation method based on hybrid attention and multi-scale supervision according to claim 1, characterized in that the loss function during the training process of the MAS-ConvUNeXt liver tumor segmentation model is a cross-entropy loss function.