WO2024031219A1

WO2024031219A1 - Image segmentation model training method, image segmentation method, and apparatus

Info

Publication number: WO2024031219A1
Application number: PCT/CN2022/110779
Authority: WO
Inventors: 廖湘湘; 许灿强; 陈必超; 李美; 黎宇翔; 张勇
Original assignee: 深圳华大生命科学研究院
Priority date: 2022-08-08
Filing date: 2022-08-08
Publication date: 2024-02-15

Abstract

Provided in the present application are an image segmentation model training method, an image segmentation method, an apparatus, and a device. The training method comprises: acquiring medical images for training and segmentation results of said medical images; and according to said medical images and the segmentation results of said medical images, training an image segmentation model until a preset condition is met, and obtaining the trained image segmentation model. The model comprises an encoder module and a decoder module; the encoder module comprises, connected in sequence, a first convolutional layer, a pooling layer and a plurality of efficient pyramid split attention units; each efficient pyramid split attention unit comprises, connected in sequence, a second convolutional layer, a pyramid squeeze attention module and a third convolutional layer. By using the pyramid squeeze attention module in the encoder module, the feature extraction capability of the encoder module is improved, so that the image segmentation model can pay more attention to a target object having a saliency feature, thus improving the segmentation accuracy of the image segmentation model.

Description

An image segmentation model training method, image segmentation method and device

Technical field

This application belongs to the field of artificial intelligence technology, and specifically relates to a training method of an image segmentation model, an image segmentation method, device and equipment.

Background technique

Currently, segmenting target objects in medical images is one of the important fields in medical image processing. For example, cell segmentation is the basic prerequisite for identifying and counting cell images.

In related technologies, segmentation of target objects in medical images can be implemented using deep learning-based segmentation algorithms. However, current segmentation algorithms based on deep learning cannot handle the boundaries of target objects well, resulting in inaccurate segmentation results.

Contents of the invention

In view of this, embodiments of the present application provide a training method for an image segmentation model, an image segmentation method, apparatus and equipment, so as to improve the segmentation accuracy of target objects.

In order to solve the above problems, the technical solutions provided by the embodiments of this application are as follows:

A method for training an image segmentation model, the method comprising:

Obtaining a medical image to be trained and a segmentation result of the medical image to be trained;

Train the image segmentation model according to the medical image to be trained and the segmentation results of the medical image to be trained until the preset conditions are reached, and a trained image segmentation model is obtained;

The model includes an encoder module and a decoder module. The encoder module includes a first convolution layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit It includes the second convolutional layer, the pyramid squeeze attention module and the third convolutional layer connected in sequence.

In a possible implementation, the pyramid squeeze attention module includes:

The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group, generate N feature maps, merge the N feature maps, and generate a merged feature map. ;N is a positive integer;

An attention unit is used to perform an attention operation on the merged feature map and generate a feature map after the attention operation;

The activation unit uses the softmax function to activate the feature map after the attention operation, performs a dot multiplication operation with the merged feature map, and outputs the final feature map.

In a possible implementation, the obtaining of the medical image to be trained and the segmentation result of the medical image to be trained includes:

Obtain medical images to be trained;

Using a preset image segmentation algorithm to extract the target object, target object outline and image background in the medical image to be trained;

The classification label that each pixel of the medical image to be trained belongs to the target object, the outline of the target object, or the image background is determined as a segmentation result of the medical image to be trained.

In a possible implementation, the image segmentation model is trained according to the medical image to be trained and the segmentation results of the medical image to be trained until a preset condition is reached, and a trained image segmentation model is obtained, including :

Input the medical image to be trained into an image segmentation model to obtain the prediction classification results of each pixel in the medical image to be trained;

The loss value is calculated according to the predicted classification result of each pixel in the medical image to be trained and the classification label of each pixel in the medical image to be trained belonging to the target object, the target object outline or the image background. The loss value is used to train the image segmentation model;

Repeatedly executing the input of the medical image to be trained into the image segmentation model to obtain the prediction classification results of each pixel in the medical image to be trained and subsequent steps until the preset conditions are reached and the image segmentation model that has been trained is obtained;

Wherein, when calculating the loss value, the loss weight when the classification label is the outline of the target object is greater than the loss weight when the classification label is the target object, and the loss weight when the classification label is the outline of the target object is greater than the loss weight when the classification label is the image background. loss weight.

An image segmentation method, the method includes:

The medical image to be segmented is input into the image segmentation model, and the initial segmentation result of each pixel in the medical image to be segmented is obtained. The initial segmentation result includes belonging to the target object, the outline of the target object, or the image background; the image segmentation model is based on It is trained by the above training method of image segmentation model;

The pixel points that the initial segmentation result belongs to the target object or the outline of the target object are determined as pixel points that belong to the target object, and the segmentation result of the medical image to be segmented is output.

In a possible implementation, the method further includes:

A watershed algorithm is used to filter target objects with an area smaller than a threshold in the segmentation results of the medical image to be segmented, and/or to repair incomplete target objects.

An image segmentation model training device, the device includes:

An acquisition unit, configured to acquire a medical image to be trained and a segmentation result of the medical image to be trained;

A training unit configured to train an image segmentation model based on the medical image to be trained and the segmentation results of the medical image to be trained until a preset condition is reached, and a trained image segmentation model is obtained;

An image segmentation device, the device includes:

An input unit is used to input the medical image to be segmented into the image segmentation model, and obtain the initial segmentation result of each pixel in the medical image to be segmented, where the initial segmentation result includes the target object, the target object outline, or the image background; The image segmentation model is trained according to the above image segmentation model training method;

A determination unit configured to determine the pixel points of the initial segmentation result belonging to the target object or the outline of the target object as pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

A training device for an image segmentation model, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above Training method for image segmentation model.

An image segmentation device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above image segmentation is implemented. method.

A computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are run on a terminal device, the terminal device causes the terminal device to execute the training method of the image segmentation model as mentioned above, or execute as The above image segmentation method.

It can be seen that the embodiments of the present application have the following beneficial effects:

The image segmentation model generated by training in the embodiment of this application consists of an encoder module and a decoder module. Among them, the encoder module includes a sequentially connected first convolution layer, pooling layer, and multiple efficient pyramid segmentation attention units. The efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeeze attention module and a third convolutional layer that are connected in sequence. That is, the pyramid squeeze attention module is used in the encoder module to improve the feature extraction capability of the encoder module, making the image segmentation model pay more attention to target objects with salient features, thus improving the segmentation accuracy of the image segmentation model.

Description of drawings

Figure 1 is a schematic diagram of an application scenario provided by an embodiment of this application;

Figure 2 is a flow chart of the training method of the image segmentation model provided by the embodiment of the present application;

Figure 3 is a schematic structural diagram of the image segmentation model in the embodiment of the present application;

Figure 4 is a schematic structural diagram of an efficient pyramid segmentation attention unit in an embodiment of the present application;

Figure 5 is a schematic structural diagram of the image segmentation model in the embodiment of the present application;

Figure 6 is a schematic structural diagram of the pyramid squeeze attention module in the embodiment of the present application;

Figure 7 is a flow chart of the image segmentation method provided in the embodiment of the present application;

Figure 8 is a diagram showing the segmentation effect of the image segmentation model of the embodiment of the present application on a mouse brain image;

Figure 9 is a diagram of the segmentation effect of the image segmentation model on monkey brain images according to the embodiment of the present application;

Figure 10 is a diagram showing the segmentation effect of the image segmentation model according to the embodiment of the present application on a salamander brain image;

Figure 11 shows the segmentation results of various algorithms on mouse brain images;

Figure 12 is a schematic structural diagram of a training device for an image segmentation model provided by an embodiment of the present application;

Figure 13 is a schematic structural diagram of an image segmentation device provided by an embodiment of the present application.

Detailed ways

In order to make the above objects, features and advantages of the present application more obvious and understandable, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation modes.

In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

Currently, segmenting target objects in medical images is one of the important fields in medical image processing, such as segmenting cells in medical images, segmenting organs and tissues in medical images, etc. However, current segmentation algorithms based on deep learning cannot handle the boundaries of target objects well when segmenting target objects, resulting in inaccurate segmentation results.

Taking cell segmentation as an example to illustrate, cell segmentation is the basic prerequisite for identifying and counting cell images. Due to the complexity of cell segmentation images, uneven illumination of microscope images, and grayscale changes of the target object itself, there are still some problems in the segmented images, such as cell adhesion and overlap.

Based on this, embodiments of the present application provide an image segmentation model training method, image segmentation method, device and equipment. The image segmentation model generated by training consists of an encoder module and a decoder module. Among them, the encoder module includes a sequentially connected first convolution layer, pooling layer, and multiple efficient pyramid segmentation attention units. The efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeeze attention module and a third convolutional layer that are connected in sequence. That is, a pyramid squeeze attention module is used in the encoder module to improve the feature extraction capability of the encoder module and make the image segmentation model pay more attention to target objects with salient features. Based on the deep learning attention mechanism, the target objects in medical images are segmented, which improves the accuracy of image segmentation.

In order to facilitate understanding of an image segmentation model training method and image segmentation method provided by embodiments of the present application, description is given below with reference to the scene example shown in Figure 1 . Refer to Figure 1, which is a schematic framework diagram of an exemplary application scenario provided by an embodiment of the present application. As an optional example, the method can be applied to terminal devices or servers.

In practical applications, the image segmentation model can be trained based on the medical images to be trained and the segmentation results of the medical images to be trained. In one possible implementation, the segmentation results of the medical image to be trained can be considered as label data, which can include classification labels indicating that each pixel in the medical image to be trained belongs to the target object, the outline of the target object, or the image background. By inputting the medical image to be trained into the image segmentation model, the prediction and classification results of each pixel in the medical image to be trained can be obtained. The prediction classification results of each pixel in the medical image to be trained and the segmentation results of the medical image to be trained are used to calculate the loss value, and the image segmentation model is adjusted using the loss value. Then, the next medical image to be trained and the segmentation result of the medical image to be trained will be used to continue to adjust the image segmentation model until the training is completed, and the trained image segmentation model is obtained.

Those skilled in the art can understand that the schematic framework diagram shown in FIG. 1 is only an example in which the embodiments of the present application can be implemented. The scope of application of the embodiments of this application is not limited by any aspect of this framework.

Based on the above description, the training method of the image segmentation model and the image segmentation method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Refer to Figure 2, which is a flow chart of an image segmentation model training method provided by an embodiment of the present application. As shown in Figure 2, the image segmentation model training method may include steps S201-S202:

S201: Obtain the medical image to be trained and the segmentation results of the medical image to be trained.

The medical images to be trained are images used to train the image segmentation model. They can be images under a microscope or images scanned by medical equipment, such as X-ray images, CT images, etc. The type of medical images to be trained can be selected based on the target objects that the image segmentation model needs to segment. For example, if the target object is a cell, then the medical image to be trained is an image under a microscope; if the target object is an organ or tissue, then the medical image to be trained is an image scanned by a medical device. The embodiment of this application does not limit the type of medical images to be trained.

The segmentation results of the medical images to be trained can be considered as the label data for training the image segmentation model. The segmentation results of the medical image to be trained may include classification labels indicating that each pixel of the medical image to be trained belongs to the target object, the outline of the target object, or the image background. The segmentation results of the medical images to be trained can be manually annotated. However, in order to improve the efficiency of obtaining label data, the segmentation results of the medical images to be trained can also be obtained through a preset image segmentation algorithm.

In one possible implementation, the specific implementation of S201 to obtain the medical image to be trained and the segmentation results of the medical image to be trained may include A1-A3:

A1: Obtain the medical images to be trained.

A2: Use the preset image segmentation algorithm to extract the target object, target object outline and image background in the medical image to be trained.

After obtaining the medical image to be trained, the preset image segmentation algorithm can be used to segment the medical image to be trained, and it is obtained that each pixel in the medical image to be trained belongs to the target object, the outline of the target object, or the image background. For example, if the medical image to be trained is an image under a microscope including cells, the deepcell and cellprofiler algorithms can be used to process the medical image to be trained, and extract the cells, cell outlines, and image background in the medical image to be trained.

A3: Determine the classification label that each pixel of the medical image to be trained belongs to the target object, the outline of the target object, or the image background as the segmentation result of the medical image to be trained.

After segmenting the medical image to be trained, the classification label that each pixel of the medical image to be trained belongs to the target object, the outline of the target object, or the image background can be determined as the segmentation result of the medical image to be trained, that is, the label data of the training image segmentation model is obtained. .

By determining the segmentation results of the medical images to be trained by preset image segmentation algorithms, segmentation results corresponding to a large number of medical images to be trained can be quickly obtained.

S202: Train the image segmentation model based on the medical image to be trained and the segmentation results of the medical image to be trained until the preset conditions are reached, and the trained image segmentation model is obtained.

By inputting the medical image to be trained into the image segmentation model, the image prediction segmentation result can be obtained. The image prediction segmentation result is compared with the segmentation result of the medical image to be trained, and the loss value is calculated. The image segmentation model can be adjusted according to the loss value, specifically adjusting the image. Model parameters for the segmentation model.

After adjusting the image segmentation model, it is necessary to determine whether the preset conditions are met. When the preset conditions are reached, the image segmentation model training ends, and the trained image segmentation model is obtained. When the preset conditions are not reached, the medical image to be trained is input into the image segmentation model again, and the image segmentation model is adjusted according to the loss value until the preset conditions are reached. As an optional example, the preset condition is reaching the preset training times. As another optional example, the preset condition is that the loss value reaches a preset loss threshold. The preset training times and preset loss threshold can be set according to the actual situation, and there are no restrictions here.

In a possible implementation, S202 trains the image segmentation model based on the medical image to be trained and the segmentation results of the medical image to be trained until the preset conditions are reached. The specific implementation of obtaining the trained image segmentation model may include B1- B3:

B1: Input the medical image to be trained into the image segmentation model and obtain the prediction classification results of each pixel in the medical image to be trained.

By inputting the medical image to be trained into the image segmentation model, the image prediction segmentation result can be obtained, which specifically can be the prediction classification result of each pixel in the medical image to be trained. Predicted classification results include belonging to the target object, target object outline, or image background.

B2: Calculate the loss value based on the predicted classification results of each pixel in the medical image to be trained and the classification label of each pixel in the medical image to be trained belonging to the target object, target object outline, or image background, and train the image segmentation model based on the loss value.

Compare the predicted classification results pixel by pixel in the medical image to be trained, with the classification label that the pixel belongs to the target object, target object outline, or image background. Use the comparison results of each pixel in the medical image to be trained to calculate the loss value, and calculate the loss value based on the loss value. Image segmentation model is trained. That is, the model parameters of the image segmentation model are adjusted according to the loss value.

B3: Repeat the input of the medical image to be trained into the image segmentation model to obtain the prediction classification results of each pixel in the medical image to be trained and subsequent steps until the preset conditions are reached and the trained image segmentation model is obtained.

The medical image to be trained is input into the image segmentation model again, and the loss value is calculated based on the predicted classification results of each pixel in the medical image to be trained and the classification label of each pixel in the medical image to be trained belonging to the target object, the outline of the target object, or the image background. The image segmentation model is trained until the preset conditions are reached, and the trained image segmentation model is obtained.

Among them, when calculating the loss value, the loss weight when the classification label is the outline of the target object is greater than the loss weight when the classification label is the target object, and the loss weight when the classification label is the outline of the target object is greater than the loss weight when the classification label is the image background.

Since the contour of the target object plays an important role in image segmentation, when calculating the loss value, the loss weight of the target object contour should be greater than the loss weight of the target object and the image background. For example, the weight loss of the target object and the image background are 0.2, and the loss weight of the target object outline is 0.6. In practical applications, the loss function for the target object and image background can be Cross Entropy, and the loss function for the contour of the target object can be focal loss.

In the embodiment of the present application, the structure of the image segmentation model is improved. Refer to Figure 3, which shows a schematic structural diagram of the image segmentation model in the embodiment of the present application.

The image segmentation model includes an encoder module and a decoder module. The encoder module includes a first convolution layer, a pooling layer, and multiple efficient pyramid segmentation attention units that are connected in sequence; the efficient pyramid segmentation attention unit includes a sequentially connected The second convolutional layer, the pyramid squeeze attention module, and the third convolutional layer.

In the embodiment of this application, the Unet network structure is improved. Based on the Resnet network structure, the last fc (Full Connection, full connection) layer is removed, and each layer in the Resnet network structure is replaced with the Pyramid Squeezing Attention Module (PSA Module). A 3x3 convolution of the residual block. The improved residual block is named Epsablock (ie, efficient pyramid segmentation attention unit). See Figure 4, which shows the schematic structural diagram of Epsablock. The efficient pyramid segmentation attention unit (Epsablock) includes a second convolution layer, a pyramid squeeze attention module (PSA Module) and a third convolution layer that are connected in sequence. . Among them, the second convolution layer and the third convolution layer can be 1x1 convolution (Conv 1x1).

Refer to Figure 5, which shows a schematic structural diagram of the image segmentation model in practical applications. Epsablock replaces the original residual block as an improved resnet. The improved resnet includes the first convolutional layer, the pooling layer, and multiple efficient pyramid segmentation attention units (Epsablock) connected in sequence. The improved resnet is used as the encoder of the Unet network structure, and the decoder is not modified. The improved Unet network structure is the image segmentation model of this application.

In a possible implementation, the pyramid squeeze attention module includes:

The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group, generate N feature maps, merge the N feature maps, and generate a merged feature map; N is a positive integer;

The attention unit is used to perform attention operations on the merged feature maps and generate feature maps after attention operations;

See Figure 6, which shows the original schematic diagram of the pyramid squeeze attention module (PSA module). The slicing unit first divides the input features into N groups using 1x1 convolution, for example, N is 4. Convolution operations with different convolution kernel sizes are performed on each group, and the convolution kernel sizes of each group increase sequentially. For example, the convolution kernel sizes are 3, 5, 7, and 9 in order. After convolutions of different sizes, the generated N feature maps are merged to generate a merged feature map.

The output of the segmentation unit is passed through the attention unit to obtain the channel attention value, that is, the feature map after the attention operation is obtained. The purpose of this is to obtain the attention weights of feature maps of different scales. In this way, contextual information of different scales is integrated and better pixel-level attention is generated. Finally, the activation unit performs softmax normalization on the feature map after the attention operation, performs dot product operation (channel-wise product) with the merged feature map, and outputs the final feature map.

The PSA module allows the image segmentation model to pay more attention to targets with salient features.

Based on the embodiments of the image segmentation model training method provided above, embodiments of the present application also provide an image segmentation method. Referring to Figure 7, Figure 7 is a flow chart of an image segmentation method provided by an embodiment of the present application. As shown in Figure 7, the method includes S701-S702:

S701: Input the medical image to be segmented into the image segmentation model, and obtain the initial segmentation results of each pixel in the medical image to be segmented. The initial segmentation results include belonging to the target object, the outline of the target object, or the image background; wherein, the image segmentation model is based on the above The image segmentation model is trained using the training method.

By inputting the medical image to be segmented into the above image segmentation model, the probability value that each pixel in the medical image to be segmented belongs to the target object, the outline of the target object, or the image background can be obtained. Each pixel in the medical image to be segmented is determined based on whether the probability value is greater than the threshold. The initial segmentation result is to determine whether each pixel in the medical image to be segmented belongs to the target object, the target object outline or the image background. In practical applications, the medical image to be segmented can be expanded into a three-channel (such as RGB three-channel) image, and the image segmentation model is input to obtain the initial segmentation result of each pixel in each channel image, that is, each pixel in each channel image The points belong to the target object, the outline of the target object or the image background, and then the initial segmentation results of each pixel point in each channel image are combined to determine the initial segmentation result of each pixel point in the medical image to be segmented.

The image segmentation model is trained according to the training method of the image segmentation model provided in the above embodiments of the present application. For relevant description, please refer to the above embodiments and will not be described again here.

S702: Determine pixel points that the initial segmentation result belongs to the target object or the outline of the target object as pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

Fusion of pixel points belonging to the target object or the contour of the target object to achieve segmentation of the target object, that is, determining the pixel points belonging to the target object or the contour of the target object as pixel points belonging to the target object, performing image segmentation, and outputting the medical image to be segmented Segmentation results.

In the embodiment of the present application, the watershed algorithm can also be used to filter the target objects whose area is smaller than the threshold in the segmentation results of the medical image to be segmented, and/or to repair the incomplete target objects.

By filtering target objects with an area smaller than the threshold and/or repairing incomplete target objects, the target object is corrected, the post-processing of the segmentation results of the medical image to be segmented is completed, and the accuracy of image segmentation is improved. For example, when the target object is cells, algorithms such as cell filtering and correction are used to filter very small cells and correct incomplete cells to make cell segmentation more accurate.

The image segmentation model used in the embodiments of this application adopts a pyramid squeeze attention module in the encoder module, which improves the feature extraction capability of the encoder module and enables the image segmentation model to pay more attention to target objects with salient features, thereby improving improve the segmentation accuracy of the image segmentation model.

Embodiments of the present application are applied to cell segmentation in spatiotemporal omics, as shown in Figure 8 , which demonstrates the segmentation effect of the image segmentation model of the embodiment of the present application on mouse brain images. A part of the mouse brain image above has been intercepted for display. As can be seen from the lower left image, there are cell adhesions in the mouse brain image, and some cell boundaries are seriously blurred. The lower right image can be achieved using the image segmentation model proposed in the embodiment of this application. Separate independent cells from adherent cells and provide more reliable boundaries.

Refer to Figures 9 and 10, which are respectively the segmentation results of the image segmentation model provided by the embodiment of the present application on monkey brain images and salamander brain images. Parts of the original images are cut out for display. It can be seen from the results that , the image segmentation model provided by the embodiments of this application can also process other images, indicating that the image segmentation model has generalization properties.

See Figure 11 for the segmentation results of multiple algorithms on mouse brain images. The left picture is a part of the mouse brain image. The middle picture is the segmentation result of the mouse brain image using the original Unet network. The right picture is the mouse brain image using the watershed algorithm. Compared with the segmentation results of the image segmentation model proposed in the embodiment of the present application (see Figure 8), in terms of segmentation of clear cells, the segmentation results of the Unet network and the image segmentation model proposed in the embodiment of the present application are similar, but in The Unet network's ability to process boundaries at adhesion cells is not as good as the image segmentation model proposed in the embodiments of this application. The watershed algorithm has poor ability to handle boundaries and over-segments cells, forcing cells to be divided into multiple cells.

To further compare the Unet network and the image segmentation model proposed in the embodiment of the present application, Table 1 is a comparison of the quantitative indicators of the Unet network and the image segmentation model proposed in the embodiment of the present application on five mouse brain images. Quantitative indicators include dice coefficient, IoU (Intersection over Union), precision (precision rate) and recall (recall rate).

It can be seen from the results that the image segmentation model proposed in the embodiment of this application has better segmentation performance.

Table 1

In summary, the segmentation effect of the image segmentation model proposed in the embodiment of this application is better than the Unet network and the watershed algorithm.

Based on the training method of the image segmentation model provided by the above method embodiment, the embodiment of the present application also provides a training device of the image segmentation model. The training device of the image segmentation model will be described below with reference to the accompanying drawings.

Referring to Figure 12, Figure 12 is a schematic structural diagram of an image segmentation model training device provided by an embodiment of the present application. As shown in Figure 12, the training device of the image segmentation model includes:

The acquisition unit 1201 is used to acquire the medical image to be trained and the segmentation result of the medical image to be trained;

The training unit 1202 is configured to train the image segmentation model according to the medical image to be trained and the segmentation results of the medical image to be trained until the preset conditions are reached, and the trained image segmentation model is obtained;

In a possible implementation, the pyramid squeeze attention module includes:

In a possible implementation, the acquisition unit includes:

Acquisition subunit, used to obtain medical images to be trained;

An extraction subunit is used to extract the target object, target object contour and image background in the medical image to be trained using a preset image segmentation algorithm;

Determining subunit, configured to determine the classification label that each pixel of the medical image to be trained belongs to the target object, the target object outline, or the image background as a segmentation result of the medical image to be trained.

In a possible implementation, the training unit includes:

An input subunit, used to input the medical image to be trained into an image segmentation model to obtain the prediction classification results of each pixel in the medical image to be trained;

A training subunit, configured to calculate based on the predicted classification results of each pixel in the medical image to be trained and the classification label that each pixel of the medical image to be trained belongs to the target object, the outline of the target object, or the image background. Loss value, the image segmentation model is trained according to the loss value;

Loop subunit, used to return to the input subunit and the training subunit to execute the input of the medical image to be trained into the image segmentation model, obtain the prediction classification results of each pixel in the medical image to be trained, and subsequent steps, until Reach the preset conditions and obtain the trained image segmentation model;

It should be noted that the technical details of the image segmentation model training device provided by the embodiments of the present application can be found in the relevant embodiments of the above image segmentation model training method, and will not be described again here.

Based on the image segmentation method provided by the above method embodiment, the embodiment of the present application also provides an image segmentation device. The image segmentation device will be described below with reference to the accompanying drawings.

Referring to Figure 13, Figure 13 is a schematic structural diagram of an image segmentation device provided by an embodiment of the present application. As shown in Figure 13, the image segmentation device includes:

The input unit 1301 is used to input the medical image to be segmented into the image segmentation model, and obtain the initial segmentation result of each pixel point in the medical image to be segmented, where the initial segmentation result includes the target object, the outline of the target object, or the image background; The image segmentation model is trained according to the above image segmentation model training method;

The determination unit 1302 is configured to determine the pixel points of the initial segmentation result belonging to the target object or the outline of the target object as pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

In a possible implementation, the device further includes:

A correction unit configured to use a watershed algorithm to filter target objects with an area smaller than a threshold in the segmentation results of the medical image to be segmented, and/or to repair incomplete target objects.

It should be noted that the technical details of the image segmentation device provided by the embodiments of the present application can be found in the relevant embodiments of the above image segmentation method, and will not be described again here.

In addition, embodiments of the present application also provide an image segmentation model training device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes When the computer program is used, the above-mentioned training method of the image segmentation model is implemented.

An embodiment of the present application also provides an image segmentation device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program , implement the image segmentation method as mentioned above.

Embodiments of the present application also provide a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are run on a terminal device, the terminal device executes the image segmentation model as described above. training method, or perform the image segmentation method as above.

It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. As for the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For relevant details, please refer to the description in the method section.

It should be understood that in this application, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And/or" is used to describe the relationship between associated objects, indicating that there can be three relationships. For example, "A and/or B" can mean: only A exists, only B exists, and A and B exist simultaneously. , where A and B can be singular or plural. The character "/" generally indicates that the related objects are in an "or" relationship. “At least one of the following” or similar expressions thereof refers to any combination of these items, including any combination of a single item (items) or a plurality of items (items). For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c" ”, where a, b, c can be single or multiple.

It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or sequence between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. Software modules may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.

The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

A method for training an image segmentation model, characterized in that the method includes:

Obtaining a medical image to be trained and a segmentation result of the medical image to be trained;

Train the image segmentation model according to the medical image to be trained and the segmentation results of the medical image to be trained until the preset conditions are reached, and a trained image segmentation model is obtained;

The model includes an encoder module and a decoder module. The encoder module includes a first convolution layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit It includes the second convolutional layer, the pyramid squeeze attention module and the third convolutional layer connected in sequence.
The method according to claim 1, characterized in that the pyramid squeeze attention module includes:

The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group, generate N feature maps, merge the N feature maps, and generate a merged feature map. ;N is a positive integer;

An attention unit is used to perform an attention operation on the merged feature map and generate a feature map after the attention operation;

The activation unit uses the softmax function to activate the feature map after the attention operation, performs a dot multiplication operation with the merged feature map, and outputs the final feature map.
The method according to claim 1, characterized in that said obtaining the medical image to be trained and the segmentation result of the medical image to be trained includes:

Obtain medical images to be trained;

Using a preset image segmentation algorithm to extract the target object, target object outline and image background in the medical image to be trained;

The classification label that each pixel of the medical image to be trained belongs to the target object, the outline of the target object, or the image background is determined as a segmentation result of the medical image to be trained.
The method according to claim 3, characterized in that the image segmentation model is trained according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is reached and a trained image is obtained. Segmentation models, including:

Input the medical image to be trained into an image segmentation model to obtain the prediction classification results of each pixel in the medical image to be trained;

The loss value is calculated according to the predicted classification result of each pixel in the medical image to be trained and the classification label of each pixel in the medical image to be trained belonging to the target object, the target object outline or the image background. The loss value is used to train the image segmentation model;

Repeatedly executing the input of the medical image to be trained into the image segmentation model to obtain the prediction classification results of each pixel in the medical image to be trained and subsequent steps until the preset conditions are reached and the image segmentation model that has been trained is obtained;

Wherein, when calculating the loss value, the loss weight when the classification label is the outline of the target object is greater than the loss weight when the classification label is the target object, and the loss weight when the classification label is the outline of the target object is greater than the loss weight when the classification label is the image background. loss weight.
An image segmentation method, characterized in that the method includes:

The medical image to be segmented is input into the image segmentation model, and the initial segmentation result of each pixel in the medical image to be segmented is obtained. The initial segmentation result includes belonging to the target object, the outline of the target object, or the image background; the image segmentation model is based on Obtained by training by the training method of the image segmentation model described in any one of claims 1-4;

The pixel points that the initial segmentation result belongs to the target object or the outline of the target object are determined as pixel points that belong to the target object, and the segmentation result of the medical image to be segmented is output.
The method of claim 5, further comprising:

A watershed algorithm is used to filter target objects with an area smaller than a threshold in the segmentation results of the medical image to be segmented, and/or to repair incomplete target objects.
An image segmentation model training device, characterized in that the device includes:

An acquisition unit, configured to acquire a medical image to be trained and a segmentation result of the medical image to be trained;

A training unit configured to train an image segmentation model based on the medical image to be trained and the segmentation results of the medical image to be trained until a preset condition is reached, and a trained image segmentation model is obtained;

The model includes an encoder module and a decoder module. The encoder module includes a first convolution layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit It includes the second convolutional layer, the pyramid squeeze attention module and the third convolutional layer connected in sequence.
An image segmentation device, characterized in that the device includes:

An input unit is used to input the medical image to be segmented into the image segmentation model, and obtain the initial segmentation result of each pixel in the medical image to be segmented, where the initial segmentation result includes the target object, the target object outline, or the image background; The image segmentation model is trained according to the training method of the image segmentation model described in any one of claims 1-4;

A determination unit configured to determine the pixel points of the initial segmentation result belonging to the target object or the outline of the target object as pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.
An image segmentation model training device, characterized in that it includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program , implement the training method of the image segmentation model as described in any one of claims 1-4.
An image segmentation device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the following: The image segmentation method according to any one of claims 5-6.
A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium. When the instructions are run on a terminal device, the terminal device is caused to execute any one of claims 1-4. The training method of the image segmentation model, or the image segmentation method as described in any one of claims 5-6.