CN116310329A - Skin lesion image segmentation method based on lightweight multi-scale UNet - Google Patents

Abstract

The invention provides a skin lesion image segmentation method based on a lightweight multi-scale UNet, which comprises the following steps: obtaining a skin lesion image and preprocessing; building a lightweight multi-scale UNet network structure, namely LMunet: based on an original UNet model, a multi-scale inversion residual error module is used for replacing an original convolution module in a UNet coding path, an asymmetric cavity space pyramid pooling module is added between the coding path and a decoding path, the number of channels of each layer is reduced, and original jumping connection of UNet is modified into channel addition; training the LMUNet network by utilizing the preprocessed skin lesion image; and inputting the skin lesion image to be segmented into a trained LMUNet network to obtain a segmentation result. Experimental results show that the method solves the problems of high complexity, large calculated amount and multiple parameters of the skin lesion image model based on UNet, improves the accuracy and the speed of dividing the skin lesion image, and realizes the rapid division of the skin lesion image.

Description

Skin lesion image segmentation method based on lightweight multi-scale UNet

technical field

The invention belongs to the technical field of medical image processing, in particular to a skin lesion image segmentation method based on lightweight multi-scale UNet.

Background technique

Image segmentation of skin lesions is an important part of skin diagnosis research. The traditional method is to make high-resolution images of damaged skin under a dermatoscope, which is then diagnosed by professional doctors. However, because the size and shape of the lesion area vary, manual judgment is time-consuming and laborious, and contains subjective components, which increases the difficulty of diagnosis. Early computer-aided medical image segmentation methods usually relied on edge detection, template matching techniques, and traditional machine learning techniques. These methods have achieved good results to a certain extent, but they often require complex preprocessing of the original image, which requires experienced engineers to design feature extractors and select appropriate classifiers for classification. Therefore, the generalization ability of this method is weak, and it is difficult to achieve complex multi-classification tasks.

With the advent of the era of big data and great advances in computer hardware, deep learning techniques, especially convolutional neural networks, have achieved better results than traditional methods in many tasks such as image classification and detection. In recent years, many researchers have focused on developing high-precision segmentation methods. Among them, UNet is a semantic segmentation network based on a fully convolutional network, which is suitable for medical image segmentation. attention. Since then, many neural networks have been developed on the basis of UNet, such as UNet++, LinkNet, 3DUNet, ResUNet, etc. Although the improved model has improved the segmentation effect, the model often has too many parameters and a large amount of calculation, which is not conducive to the realization of image quality. quick split.

Contents of the invention

In order to solve the above-mentioned technical defects in the prior art, the present invention proposes a skin lesion image segmentation method based on lightweight multi-scale UNet.

The technical solution for realizing the object of the present invention is: a skin lesion image segmentation method based on lightweight multi-scale UNet, comprising the following steps:

Step 1. Obtain the skin lesion image and perform preprocessing;

Step 2. Establish a lightweight multi-scale UNet network structure, namely LMUNet: based on the original UNet model, use a multi-scale inverted residual module to replace the original convolution module in the UNet encoding path, and add Asymmetric empty space pyramid pooling module, while reducing the number of channels in each layer, and modifying UNet's original skip connection to channel addition;

Step 3, utilize the preprocessed skin lesion image to train the LMUNet network;

Step 4. Input the image of the skin lesion to be segmented into the trained LMUNet network to obtain the segmentation result.

Preferably, the preprocessing of the skin lesion image in step 1 specifically includes:

Unified image size, data enhancement and data set division, wherein the unified image size is to unify the acquired skin lesion image size by scaling or cropping; data enhancement adopts rotation and flip image geometric transformation method and histogram equalization One or more; Data set division refers to dividing the image after the data enhancement operation into a training set, a verification set and a test set.

Preferably, the establishment of a lightweight multi-scale UNet network structure, namely LMUNet, as described in step 2, is as follows:

LMUNet network structure includes encoding path, decoding path and multi-scale information fusion module;

The encoding path consists of a multi-scale inverted residual module and a 2×2 maximization pooling module, which is responsible for extracting features from the input image and reducing the size of the image, reducing the amount of redundant parameters; the decoding path consists of a deconvolution layer and a standard convolution Layers are responsible for restoring feature map information; the multi-scale information fusion module is located between the encoding path and the decoding path, and is used to fuse features at different scales and enrich context information.

Preferably, the encoding path includes a multi-scale inversion residual module and a 2×2 maximization pooling module; wherein, the multi-scale inversion residual module is responsible for extracting feature information of images at multiple scales, and the 2×2 maximization pooling module The module can compress the image, extract the main features of the image and reduce the amount of network redundant parameters.

The multi-scale inversion residual module includes a first 3×3 depth convolution module, a second 3×3 depth convolution module, a 3×3 depth hole convolution module and a 1×1 point convolution module, the first 3× The 3-depth convolution module includes 3×3 depth convolution, batch normalization layer and ReLU6 activation function. The input information passes through 3×3 depth convolution, batch normalization layer and ReLU6 activation function once, and the obtained result is consistent with the input information Perform channel splicing, and input the spliced information into the second 3×3 depth convolution module and the 3×3 depth hole convolution module, the second 3×3 depth convolution module includes 3×3 depth convolution, batch A normalization layer and a ReLU6 activation function; the 3×3 depth hole convolution module includes a 3×3 depth hole convolution with a hole rate of 2, a batch normalization layer and a ReLU6 activation function;

The results obtained by the second 3×3 depth convolution module and the 3×3 depth hole convolution module are added and then input into the 1×1 point convolution module. The 1×1 point convolution module includes a 3×3 depth convolution module Product, batch normalization layer and ReLU6 activation function, after 3×3 depth convolution, batch normalization layer and ReLU6 activation function, the result obtained is added to the input information at the beginning to obtain different scales characteristic information.

Preferably, the multi-scale information fusion module is composed of an asymmetric hollow space pyramid pooling module, including 1×1 point convolution and 3 parallel branches, and the 3 parallel branches are:

Branch 1: It consists of 3×1 depth asymmetric convolution and 1×3 depth asymmetric convolution;

Branch 2: It consists of a 3×1 depth asymmetric atrous convolution with a dilation rate of 2 and a 1×3 depth asymmetric atrous convolution;

Branch 3: It consists of a 3×1 depth asymmetric hole convolution and a 1×3 depth asymmetric hole convolution with a hole rate of 3;

The input information goes through 1×1 point convolution, batch normalization layer and ReLU6 activation function, the number of channels is reduced to half of the original, and after three parallel branches, the obtained result and the information of the input branch are channel spliced, and finally After 1×1 point convolution for feature information fusion, the fused multi-scale feature information is obtained;

Preferably, step 3 utilizes the preprocessed skin lesion image to train the LMUNet network, and the loss function used is a cross-entropy function, and the function form is:

In the formula, H(P, Q represents cross entropy, P( _xi ) represents the real probability distribution, and Q( _xi ) represents the predicted probability distribution.

Compared with the prior art, the present invention has the following significant advantages: the present invention is based on the UNet model, and improves the UNet model by introducing a multi-scale inverted residual module, an asymmetric empty space pyramid pooling module, and reducing the number of channels. Experimental results show that compared with other segmentation models, the LMUNet model proposed by the present invention has superior performance and is very lightweight, and can achieve a better segmentation effect with only a small amount of calculation.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Description of drawings

The drawings are for the purpose of illustrating specific embodiments only and are not to be considered as limitations of the invention, and like reference numerals refer to like parts throughout the drawings.

Fig. 1 is the overall structural diagram of LMUNet network among the present invention.

Fig. 2 is a structural diagram of the multi-scale inverted residual module in the present invention.

Fig. 3 is a block diagram of the multi-scale information fusion module in the present invention.

Fig. 4 is a schematic flow chart of the method of the present invention.

Detailed ways

It is easy to understand that, according to the technical solution of the present invention, those skilled in the art can imagine various implementations of the present invention without changing the essence and spirit of the present invention. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the entirety of the present invention or as a limitation or limitation on the technical solution of the present invention. Rather, these embodiments are provided to enable those skilled in the art to more thoroughly understand the present invention. Preferred embodiments of the present invention will be specifically described below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of the application and are used together with the embodiments of the present invention to explain the innovative concept of the present invention.

The concept of the present invention is, as shown in Figures 1 to 4, a skin lesion image segmentation method based on lightweight multi-scale UNet, the specific steps of the method are as follows:

Step 1. Obtain the skin lesion image and perform preprocessing;

Step 2. Establish a lightweight multi-scale UNet network structure, namely LMUNet: based on the original UNet model, use a multi-scale inverted residual module to replace the original convolution module in the UNet encoding path, and add Asymmetric empty space pyramid pooling module, while reducing the number of channels in each layer, and modifying UNet's original skip connection to channel addition;

Step 3, utilize the preprocessed skin lesion image to train the LMUNet network;

Step 4. Input the image of the skin lesion to be segmented into the trained LMUNet network to obtain the segmentation result.

In a further embodiment, the specific method of acquiring and preprocessing the skin lesion image is: unifying the image size, data enhancement, and data set division, wherein the unifying image size is to scale or crop the acquired skin lesion image size. Unification; data enhancement adopts one or more of geometric transformation methods such as rotating and flipping images, and image enhancement methods such as histogram equalization; data set division divides the image after data enhancement operation into training set, verification set and test set.

In a further embodiment, as shown in FIG. 1 , the lightweight multi-scale UNet network structure, namely LMUNet, includes an encoding path, a decoding path, and a multi-scale information fusion module. The encoding path consists of a multi-scale inverted residual module and a 2×2 max pooling module, where the max pooling module performs a total of 4 downsampling; the decoding path consists of a 3×3 standard convolution block and a 2×2 deconvolution , where the deconvolution layer performs a total of 4 times of upsampling; the multi-scale information fusion module is located between the encoding path and the decoding path, and is used to fuse features at different scales and enrich context information.

Specifically, the multi-scale inverted residual module structure is shown in Figure 2, specifically, the multi-scale inverted residual module structure consists of 3×3 depth convolution, 3×3 depth hole convolution, 1×1 point convolution, Batch normalization layer and ReLU6 activation function. The input information of this module first passes through 3×3 depth convolution, batch normalization layer and ReLU6 activation function. Parallel branches:

Branch 1: It consists of 3×3 depth convolution;

Branch 2: Consists of 3×3 deep hole convolution with a hole rate of 2;

Add the results obtained by the two branches and then go through 3×3 depth convolution, batch normalization layer and ReLU6 activation function, and finally add the obtained results to the input information at the beginning. Using this module, feature information at different scales can be obtained.

Further, the structure of the multi-scale information fusion module is shown in Figure 3, specifically, the multi-scale information fusion module is composed of an asymmetric hollow space pyramid pooling module, including three parallel branches:

Branch 1: It consists of 3×1 depth asymmetric convolution and 1×3 depth asymmetric convolution;

Branch 2: It consists of a 3×1 depth asymmetric atrous convolution with a dilation rate of 2 and a 1×3 depth asymmetric atrous convolution;

Branch 3: It consists of a 3×1 depth asymmetric hole convolution and a 1×3 depth asymmetric hole convolution with a hole rate of 3;

The input information first passes through 1×1 point convolution, batch normalization layer and ReLU6 activation function, reducing the number of channels to half of the original, and then passes through 3 parallel branches, and combines the results obtained by the 3 branches with the information of the input branch Carry out channel splicing, and finally perform feature information fusion through 1×1 point convolution. This module can be used to fuse feature information at multiple scales to enhance model capabilities.

During the training process, the network models are implemented on the Pytorch platform, and the version is Pytorch1.11.0. The hyperparameters during the training process are as follows: the learning rate is set to 0.01, the number of training iterations is set to 100, and the number of samples for each training is 4. The optimizer uses the stochastic gradient descent method to update the network parameters to speed up the convergence. The loss function used is the cross entropy function, and the function form is:

In the formula, H(P, Q represents cross entropy, P( _xi ) represents the real probability distribution, and Q( _xi ) represents the predicted probability distribution.

Table 1 The comparison of segmentation results between the method of the present invention and the partial convolutional neural network method

From the segmentation results in Table 1, it can be seen that compared with other convolutional neural networks in Table 1, the present invention has superior performance and is very lightweight, reduces the amount of parameters and calculations, and improves the accuracy and speed of skin lesion image segmentation.

The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto.

Any changes or substitutions that can be easily conceived by any person skilled in the art within the technical scope disclosed in the present invention shall fall within the protection scope of the present invention.

It should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline the present disclosure and to assist those skilled in the art in understanding its various aspects, various features of the invention are sometimes described in the context of a single embodiment, or with reference to A single graph is described. However, the present invention should not be interpreted that the features included in the exemplary embodiments are all essential technical features of the patent claims.

It should be understood that the modules, units, components, etc. included in the device of one embodiment of the present invention can be adaptively changed so as to be arranged in a device different from that of the embodiment. Different modules, units or components included in the device of the embodiment can be combined into one module, unit or component, or they can be divided into multiple sub-modules, sub-units or sub-components.

Claims (6)

1. a skin lesion image segmentation method based on lightweight multi-scale UNet, is characterized in that, comprises the following steps: Step 1. Obtain the skin lesion image and perform preprocessing; Step 2. Establish a lightweight multi-scale UNet network structure, namely LMUNet: based on the original UNet model, use a multi-scale inverted residual module to replace the original convolution module in the UNet encoding path, and add Asymmetric empty space pyramid pooling module, while reducing the number of channels in each layer, and modifying UNet's original skip connection to channel addition; Step 3, utilize the preprocessed skin lesion image to train the LMUNet network; Step 4. Input the image of the skin lesion to be segmented into the trained LMUNet network to obtain the segmentation result. 2. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1, wherein the step 1 preprocessing the skin lesion image specifically includes: Unified image size, data enhancement and data set division, wherein the unified image size is to unify the acquired skin lesion image size by scaling or cropping; data enhancement adopts rotation and flip image geometric transformation method and histogram equalization One or more; Data set division refers to dividing the image after the data enhancement operation into a training set, a verification set and a test set. 3. the skin lesion image segmentation method based on light-weight multi-scale UNet according to claim 1, is characterized in that, the establishment of light-weight multi-scale UNet network structure i.e. LMUNet described in step 2 is specifically as follows: LMUNet network structure includes encoding path, decoding path and multi-scale information fusion module; The encoding path consists of a multi-scale inverted residual module and a 2×2 maximization pooling module, which is responsible for extracting features from the input image and reducing the size of the image, reducing the amount of redundant parameters; the decoding path consists of a deconvolution layer and a standard convolution Layers are responsible for restoring feature map information; the multi-scale information fusion module is located between the encoding path and the decoding path, and is used to fuse features at different scales and enrich context information. 4. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 3, wherein the encoding path includes a multi-scale inverted residual module and a 2×2 maximization pooling module; wherein, multiple The scale inversion residual module is responsible for extracting the feature information of the image at multiple scales, while the 2×2 maximization pooling module can compress the image, extract the main features of the image and reduce the amount of network redundant parameters. The multi-scale inversion residual module includes a first 3×3 depth convolution module, a second 3×3 depth convolution module, a 3×3 depth hole convolution module and a 1×1 point convolution module, the first 3× The 3-depth convolution module includes 3×3 depth convolution, batch normalization layer and ReLU6 activation function. The input information passes through 3×3 depth convolution, batch normalization layer and ReLU6 activation function once, and the obtained result is consistent with the input information Perform channel splicing, and input the spliced information into the second 3×3 depth convolution module and the 3×3 depth hole convolution module, the second 3×3 depth convolution module includes 3×3 depth convolution, batch A normalization layer and a ReLU6 activation function; the 3×3 depth hole convolution module includes a 3×3 depth hole convolution with a hole rate of 2, a batch normalization layer and a ReLU6 activation function; The results obtained by the second 3×3 depth convolution module and the 3×3 depth hole convolution module are added and then input into the 1×1 point convolution module. The 1×1 point convolution module includes a 3×3 depth convolution module Product, batch normalization layer and ReLU6 activation function, after 3×3 depth convolution, batch normalization layer and ReLU6 activation function, the result obtained is added to the input information at the beginning to obtain different scales characteristic information. 5. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1, wherein the multi-scale information fusion module is composed of an asymmetric cavity space pyramid pooling module, including 1×1 point volume Product and three parallel branches, the three parallel branches are: Branch 1: It consists of 3×1 depth asymmetric convolution and 1×3 depth asymmetric convolution; Branch 2: It consists of a 3×1 depth asymmetric atrous convolution with a dilation rate of 2 and a 1×3 depth asymmetric atrous convolution; Branch 3: It consists of a 3×1 depth asymmetric atrous convolution with a dilation rate of 3 and a 1×3 depth asymmetric atrous convolution. The input information goes through 1×1 point convolution, batch normalization layer and ReLU6 activation function, the number of channels is reduced to half of the original, and after three parallel branches, the obtained result and the information of the input branch are channel spliced, and finally After 1×1 point convolution for feature information fusion, the fused multi-scale feature information is obtained; 6. The skin lesion image segmentation method based on lightweight multi-scale UNet according to claim 1, wherein step 3 utilizes the preprocessed skin lesion image to train the LMUNet network, and the loss function used is a cross entropy function, The function form is:

In the formula, H(P,Q) represents the cross entropy, P( _xi ) represents the real probability distribution, Q( _xi ) represents the predicted probability distribution, _xi is a variable, and n is the number of variables.

Images