CN116205936A - An Image Segmentation Method Introducing Spatial Information and Attention Mechanism - Google Patents

Abstract

The invention relates to an image segmentation method for introducing spatial information and an attention mechanism, belonging to the field of image segmentation. The method specifically comprises the following steps: s1: data preparation stage: preprocessing a brain medical image, and cutting out an image block from the preprocessed image; s2: feature encoding stage: extracting image features by pre-activated 3D convolution; s3: feature decoding stage: and restoring the original image size of the feature map obtained in the encoding stage through deconvolution and a attention mechanism with position encoding, and completing the image segmentation process. According to the invention, the attention mechanism is used for paying attention to the space information, so that the network segmentation performance is improved.

Description

An Image Segmentation Method Introducing Spatial Information and Attention Mechanism

technical field

The invention belongs to the field of image segmentation and relates to an image segmentation method introducing spatial information and attention mechanism.

Background technique

With the rapid development of human medical imaging technologies such as Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), medical images are playing an increasingly important role in clinical medical diagnosis. Medical image segmentation can better provide scientific reference for medical staff to judge and diagnose the cause of the disease, which will greatly reduce the misdiagnosis rate caused by insufficient visual resolution of human beings or insufficient clinical experience of medical staff, and further improve the accuracy of medical images. utilization rate. U-Net has been proven to be very effective in medical image processing tasks. In medical images, there is a 3D image with complex anatomical structures, and the convolution-based codec structure cannot make full use of the spatial information of 3D images.

Contents of the invention

In view of this, the purpose of the present invention is to provide an image segmentation method that introduces spatial information and an attention mechanism, by using 3D relative position coding and attention mechanism to fully mine the spatial information of 3D images, and on the decoding path, more accurate The semantic information of the restored image can be improved to improve the segmentation accuracy of the model.

To achieve the above object, the present invention provides the following technical solutions:

An image segmentation method that introduces spatial information and attention mechanism. This method designs a self-attention segmentation network assisted by relative position encoding. In the encoding stage, image features are extracted by pre-activated 3D convolution. In the decoding stage, The image size is gradually restored by deconvolution, and the image features are restored by the NonLocal self-attention module embedded with relative position encoding. This method includes the following steps:

S1: Data preparation stage: preprocessing brain medical images, cropping image blocks from preprocessed images;

S2: Feature encoding stage: extract image features through pre-activated 3D convolution;

S3: Feature decoding stage: The feature map obtained in the encoding stage is restored to the original image size through deconvolution and attention mechanism with position encoding, and the image segmentation process is completed.

Further, it is characterized in that the step S1 includes the following steps:

S11: cropping the 3D medical image, cutting off the background area with a gray value of 0 along the plane formed by any two axes;

S12: Use Z-Score normalization on the cropped image, so that the mean value of the gray distribution of the image is 0, and the standard deviation is 1, so that it obeys the normal distribution;

S13: Cut the cropped image into image blocks of 32×32×32 in size, randomly select a block as the input of the feature encoding stage in step S2; if it is multi-modal data, all modes The state data are spliced along the channel dimension to form a multi-channel image as the input of the feature encoding stage in step S2.

Further, the step S2 includes the following steps:

S21: Using common 3D convolution extraction to perform feature extraction on the cropped 3D image to obtain a 32×32×32 feature map;

S22: use a 3D convolution with a step size of 2 to perform a downsampling operation on the feature map of S21;

S23: Repeat operations S21 and S22 to finally obtain a 4×4×4 feature map.

Further, the step S3 specifically includes the following steps:

S31: For the 3D feature map obtained in the encoding stage, calculate the relative positions of other pixels with all pixels as the origin;

S32: Embed the position encoding generated in S31 into the NonLocal self-attention mechanism, and perform feature fusion on the feature map in the encoding stage;

S33: Use deconvolution to upsample the feature map, and then repeat S32;

S34: Repeat step S33 twice to realize image segmentation.

The beneficial effect of the present invention is that spatial information is introduced through relative position encoding and embedded into the self-attention mechanism, so that the learning of weights in the attention mechanism not only depends on grayscale information but also depends on position information. Attention mechanism is used to pay attention to spatial information and improve network segmentation performance.

Other advantages, objects and features of the present invention will be set forth in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the investigation and research below, or can be obtained from Taught in the practice of the present invention. The objects and other advantages of the invention may be realized and attained by the following specification.

Description of drawings

In order to make the purpose of the present invention, technical solutions and advantages clearer, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

Fig. 1 is the network structure diagram of the image segmentation model among the present invention;

Fig. 2 is a schematic diagram of a relative position encoding structure in the present invention;

Fig. 3 is an upsampling module of the network in the present invention.

Detailed ways

Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments are only schematically illustrating the basic concept of the present invention, and the following embodiments and the features in the embodiments can be combined with each other in the case of no conflict.

Wherein, the accompanying drawings are for illustrative purposes only, and represent only schematic diagrams, rather than physical drawings, and should not be construed as limiting the present invention; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, Enlargement or reduction does not represent the size of the actual product; for those skilled in the art, it is understandable that certain known structures and their descriptions in the drawings may be omitted.

In the drawings of the embodiments of the present invention, the same or similar symbols correspond to the same or similar components; , "front", "rear" and other indicated orientations or positional relationships are based on the orientations or positional relationships shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred devices or elements must It has a specific orientation, is constructed and operated in a specific orientation, so the terms describing the positional relationship in the drawings are for illustrative purposes only, and should not be construed as limiting the present invention. For those of ordinary skill in the art, the understanding of the specific meaning of the above terms.

Please refer to accompanying drawings 1 to 3. The present invention provides an image segmentation method that introduces spatial information and attention mechanism. In this embodiment, it is assumed that it is used to segment brain tissue images. This method designs a relative position encoding The auxiliary self-attention segmentation network, the network structure diagram is shown in Figure 1. In the encoding stage, image features are extracted by pre-activated 3D convolution, and in the decoding stage, the image size is gradually restored by deconvolution, and restored by the NonLocal self-attention module embedded with relative position encoding (as shown in Figure 3) image features. The method comprises the steps of:

Step 1: Preprocess the brain medical image, and cut out a 32×32×32 image block from the preprocessed image;

Step 101: cutting the three-dimensional medical image, cutting off the background area with a gray value of 0 along the plane formed by any two axes;

Step 102: Use Z-Score normalization on the cropped image, so that the mean value of the gray distribution of the image is 0, and the standard deviation is 1, so that it obeys the normal distribution;

Step 103: Cut the cropped image into 32×32×32 image slices, randomly select a slice as the input of the model; if it is multi-modal data, all the modality data along the The channel dimensions are concatenated to form a multi-channel image as network input.

Step 2: Feature encoding stage: extract image features through pre-activated 3D convolution;

Step 201: using common 3D convolution extraction to perform feature extraction on the 3D image to obtain a 32×32×32 feature map;

Step 202: use a 3D convolution with a step size of 2 to perform a downsampling operation on the feature map in step 201;

Step 203: Repeat steps 201 and 202 to finally obtain a 4×4×4 feature map.

Step S3: Feature decoding stage: The feature map obtained in the encoding stage is restored to the original image size through deconvolution and attention mechanism with position encoding, and the image segmentation process is completed.

Step 301: For the 3D feature map obtained in the encoding stage, calculate the relative positions of other pixels with all pixels as the origin, as shown in FIG. 2 ;

Step 302: As shown in Figure 3, embed the position code generated in step 301 into the NonLocal self-attention mechanism, and perform feature fusion on the feature map in the coding stage.

Step 303: Use deconvolution to upsample the feature map, and then repeat step 302.

Step 304: Repeat step 303 twice to realize image segmentation.

The spatial information is introduced by relative position encoding and embedded into the self-attention mechanism, so that the learning of weights in the attention mechanism not only depends on the gray level information but also depends on the position information. Attention mechanism is used to pay attention to spatial information and improve network segmentation performance.

In order to verify the effect of the present invention, the following experiments have been carried out:

Based on the image segmentation method that introduces spatial information and attention mechanism, it is tested on the IBSR18 dataset. The IBSR18 dataset contains 18 training samples, and the test goal is to segment brain tissue MRI images into gray matter (GM), white matter (WM), cerebrospinal fluid (CSF) and background. 14 data samples are used as the training set, and the remaining one is used as the validation set. At the same time, method 1 using spatial attention and channel attention, method 2 using self-attention mechanism, method 3 using axial attention and method 4 of the present invention are compared. The Dice coefficient is used as the evaluation index, and its formula is as follows:

Among them, A represents the result of neural network segmentation, and B represents the gold standard given by the data set.

Table 1 shows the test results on the data set. It can be seen that the neural network based on the present invention performs better on each segmentation result on the Dice coefficient.

Table 1

CSF GM W M AVG method 1 85.88 95.30 95.05 92.08 Method 2 86.12 95.38 95.04 92.17 Method 3 86.20 95.37 95.01 92.19 Method 4 86.77 95.48 95.06 92.44

Finally, it is noted that the above embodiments are only used to illustrate the technical solutions of the present invention without limitation. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be carried out Modifications or equivalent replacements, without departing from the spirit and scope of the technical solution, should be included in the scope of the claims of the present invention.

Claims (4)

1. An image segmentation method introducing spatial information and an attention mechanism is characterized in that: comprises the following steps:

s1: data preparation stage: preprocessing a brain medical image, and cutting out an image block from the preprocessed image;

s2: feature encoding stage: extracting image features by pre-activated 3D convolution;

s3: feature decoding stage: and restoring the original image size of the feature map obtained in the encoding stage through deconvolution and a attention mechanism with position encoding, and completing the image segmentation process.

2. The method of image segmentation incorporating spatial information and attention mechanisms of claim 1, wherein: the method is characterized in that the step S1 comprises the following steps:

s11: cutting the three-dimensional medical image, and cutting off a background area with a gray value of 0 along a plane formed by any two axes;

s12: the cut image is normalized by Z-Score, the average value of gray distribution of the image is 0, and the standard deviation is 1, so that the image is subjected to normal distribution;

s13: cutting the cut image into a plurality of image cut blocks with the size of 32 multiplied by 32, and randomly selecting one cut block as an input of the feature encoding stage in the step S2; if the data are multi-mode data, all the mode data are spliced together along the channel dimension to form a multi-channel image which is used as the input of the feature encoding stage in the step S2.

3. The method of image segmentation incorporating spatial information and attention mechanisms of claim 1, wherein: the method is characterized in that the step S2 comprises the following steps:

s21: performing feature extraction on the cut three-dimensional image by using common 3D convolution extraction to obtain 32X 32 feature mapping;

s22: downsampling the feature map of S21 using a 3D convolution with a step size of 2;

s23: the operations S21 and S22 are repeated, finally obtain 4 multiplied by 4 feature map x 4.

4. The method of image segmentation incorporating spatial information and attention mechanisms of claim 1, wherein: the method is characterized in that the step S3 specifically comprises the following steps:

s31: for the 3D feature map obtained in the encoding stage, calculating the relative positions of other pixel points by taking all the pixel points as original points;

s32: embedding the position code generated in the step S31 into a non local self-attention mechanism, and carrying out feature fusion on the feature map of the coding stage;

s33: upsampling the feature map using deconvolution, followed by repeating S32;

s34: the step S33 is repeated twice to realize image segmentation.

Images