CN114463361A - Network model training method, device, equipment, medium and program product - Google Patents

Abstract

The present disclosure provides a network model training method and device, an electronic device, a non-transitory computer-readable storage medium storing computer instructions, and a computer program product, which relate to the technical field of deep learning, and in particular, to the technical field of image segmentation. The present disclosure firstly generates a probability map corresponding to the target object based on the annotation information of a plurality of first sample images for the target object; and then uses the probability map and unlabeled images to perform pre-training, so as to obtain a pre-training including high-quality network parameters. On this basis, a small number of labeled medical images are used to further train the pre-trained model, and the obtained image segmentation model has high image segmentation accuracy.

Description

Network model training method, device, equipment, medium and program product

technical field

The present disclosure relates to the technical field of deep learning, and in particular, to the technical field of image segmentation, and discloses a network model training method and device, an electronic device, a non-transitory computer-readable storage medium storing computer instructions, and a computer program product.

Background technique

The field of medical image segmentation can be mainly divided into two types of segmentation, one is structural segmentation (such as brain tissue, lungs, liver and heart, etc.), and the other is lesion segmentation. In recent years, deep learning has achieved very good results in the field of medical image segmentation, with the advantages of high robustness, higher accuracy and faster speed. Generally speaking, deep learning requires a large amount of labeled data to complete the training of the model. However, because medical images are mainly three-dimensional images, and the quality of medical images is poorer than that of traditional natural images, the contrast is difficult and time-consuming. In the field of image segmentation, the amount of labeled data is relatively small, which greatly limits the application of deep learning in the field of medical image segmentation.

SUMMARY OF THE INVENTION

The present disclosure provides at least a network model training method, apparatus, device, program product, and storage medium.

According to an aspect of the present disclosure, a network model training method is provided, including:

Generate a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object;

Based on the probability map and multiple second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain a pre-training model;

Based on the multiple third sample images and the labeling information of the target object by the third sample images, the pre-training model is trained to obtain an image segmentation model for the target object.

According to another aspect of the present disclosure, a network model training method is provided, comprising:

Generate a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object;

Based on the probability map and multiple second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image, and an image restoration model is obtained.

According to another aspect of the present disclosure, a network model training apparatus is provided, comprising:

a first atlas determination module, configured to generate a probability atlas corresponding to the target object based on the annotation information of the plurality of first sample images for the target object;

The pre-training module is used to perform model training based on the probability map and multiple second sample images, with the goal of restoring the masked image blocks in each second sample image, to obtain a pre-training model;

The segmentation model training module is used to train the pre-training model based on the multiple third sample images and the annotation information of the third sample images to the target object, and obtain the image segmentation model for the target object:

According to another aspect of the present disclosure, a network model training apparatus is provided, comprising:

A second atlas determination module, configured to generate a probability atlas corresponding to the target object based on the annotation information of the plurality of first sample images for the target object;

The recovery model training module is used to perform model training based on the probability map and the plurality of second sample images with the goal of recovering the masked image blocks in each second sample image to obtain an image recovery model.

According to another aspect of the present disclosure, there is provided an electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method in any of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method in any of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising computer programs/instructions that, when executed by a processor, implement the method in any of the embodiments of the present disclosure.

According to the technology of the present disclosure, using a probability map and an unlabeled image, that is, a second sample image for pre-training, a pre-training model containing high-quality network parameters can be obtained. The medical image, that is, the third sample image, further trains the pre-training model, and the obtained image segmentation model can determine the segmentation result of the medical image with higher segmentation accuracy.

It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of drawings

The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

1 is one of the flow charts of the network model training method according to the present disclosure;

2 is a flowchart of a training method of a pre-trained model according to the present disclosure;

3 is a schematic structural diagram of an encoder according to the present disclosure;

4 is a flowchart of a training method for an image segmentation model according to the present disclosure;

5 is the second flow chart of the network model training method according to the present disclosure;

6 is the third flow chart of the network model training method according to the present disclosure;

Fig. 7 is the fourth flow chart of the network model training method according to the present disclosure;

8 is one of the schematic structural diagrams of the network model training apparatus according to the present disclosure;

FIG. 9 is the second schematic structural diagram of the network model training apparatus according to the present disclosure;

FIG. 10 is a schematic structural diagram of an electronic device according to the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

The field of medical image segmentation can be mainly divided into two types of segmentation, one is structural segmentation (such as brain tissue, lungs, liver and heart, etc.), and the other is lesion segmentation. In recent years, deep learning has achieved very good results in the field of medical image segmentation, with the advantages of high robustness, higher accuracy and faster speed. Generally speaking, deep learning requires a large amount of labeled data to complete the training of the model. However, because medical images are mainly three-dimensional images, and the quality of medical images is poorer than that of traditional natural images, the contrast is difficult and time-consuming. In the field of image segmentation, the amount of labeled data is relatively small, which greatly limits the application of deep learning in the field of medical image segmentation.

In view of this technical defect, the present disclosure provides at least a network model training method, apparatus, device, program product, and storage medium. The present disclosure uses the probability map and unlabeled images, that is, the second sample image, for pre-training, and can obtain a pre-training model containing high-quality network parameters. That is, the third sample image further trains the pre-training model, and the obtained image segmentation model can determine the segmentation result of the medical image with higher segmentation accuracy.

The network model training method of the present disclosure will be described below through specific embodiments.

FIG. 1 shows a flowchart of a method for training a network model according to an embodiment of the present disclosure, and an executive body of this embodiment may be a device with computing capabilities. As shown in FIG. 1 , the network model training method according to the embodiment of the present disclosure may include the following steps:

S110. Generate a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object.

The above-mentioned first sample image may be a medical image with labeled information, for example, an abdominal image with labeled information corresponding to the liver. The above-mentioned labeling information may specifically include information on whether each pixel segment point in the chest image belongs to the liver.

The probability map includes the probability that each pixel in the image of the preset size belongs to the target object. The image of the preset size has the same resolution and size as the second sample image below, so that the probability map and each second sample can be combined. The image is used for model training, and a pre-trained model with high image restoration accuracy is obtained.

S120. Based on the probability map and the plurality of second sample images, perform model training with the goal of restoring the masked image blocks in each second sample image, to obtain a pre-trained model.

The above-mentioned second sample image is an image without labeling information of the target object, for example, it may be an abdominal image without labeling information corresponding to the liver. The sources of the second sample image and the first sample image may be different, but both need to include the target object, for example, both need to include the liver.

Since the second sample image does not need labeling information, the second sample image is relatively easy to obtain, the number of second sample images used to train the pre-training model is relatively large, the pre-training model can be fully trained, and its image restoration ability or image Strong reduction ability and high precision. The high image restoration ability of the pre-trained model means that it can extract more accurate image features. Based on this, the pre-trained model is further trained, and an image segmentation model with high image segmentation accuracy can be obtained.

The position of the above-mentioned target object in the corresponding image is relatively fixed. Therefore, training the pre-training model in combination with the probability atlas can improve the accuracy of the pre-training model in restoring the image block corresponding to the target object. For example, the position of the liver in the abdominal image is relatively fixed, and the probability map can more accurately characterize whether the corresponding pixel belongs to the liver. Training the pre-training model in combination with the probability map corresponding to the liver can improve the recovery of the image corresponding to the liver by the pre-training model. The precision of the block.

S130: Train the pre-training model based on the plurality of third sample images and the labeling information of the third sample images on the target object to obtain an image segmentation model for the target object.

The above-mentioned third sample image may be a medical image with labeled information, for example, may be an abdominal image with labeled information corresponding to the liver. The third sample image may be the same image as the above-mentioned first sample image, or may be a different image, which is not limited in the present disclosure

Using the annotation information of the target object by the third sample image to perform supervised learning on the image segmentation of the target object, an image segmentation model with high segmentation accuracy for the target object can be obtained.

The above-mentioned pre-training model is obtained by combining the probability map training, which can extract more accurate image features. The prior information in the multiple first sample images, that is, the information in the probability map, is fused into the pre-training model obtained by training the second sample image to improve the transfer ability of the pre-training model for image segmentation, so as to be able to Improve the accuracy of segmentation of objects or structures in images.

In some embodiments, the above-mentioned probability map can be generated by the following steps:

First, for each first sample image, based on the annotation information of the first sample image for the target object, a mask image of the first sample image for the target object is generated; Mask the image to generate a probability map corresponding to the target object.

The above mask image has the same size and resolution as the second sample image.

The above-mentioned mask image may be a binary image representing whether each pixel in the mask image is a target object. Therefore, when determining the mask image, annotation information for the target object needs to be based on the first sample image. Specifically, when the label information of a pixel indicates that the pixel belongs to the target object, the pixel value of the pixel in the mask image is 1; when the label information of the pixel indicates that the pixel does not belong to the target object , the pixel value of this pixel in the mask image is 0.

After obtaining the mask images of each first sample image, the pixel values of the pixels at the same position in each mask image are summed and then averaged to obtain the probability of whether the corresponding pixel belongs to the target object, For example, the probability of a pixel can be determined using the following formula:

In the formula, P _posion represents the probability of the pixel point at the position (x, y, z); N represents the number of mask images, (x, y, z) represents the position coordinate of the pixel point, I _{i(x, y , z)} represents the pixel value of the pixel at the position (x, y, z) in the ith mask image.

According to the above method, a relatively accurate probability of whether each pixel point belongs to the target object can be determined, and then a probability map is formed by using the probability of each pixel point. The above probability map can accurately reflect the position information of the target object in the image, and can play a good guiding role in the segmentation and restoration of the target object.

Since the resolution and size of different first sample images may be different, in order to improve the accuracy of the generated probability map, before generating the mask image of each first sample image, each first sample image can be Preprocessing is performed to unify all first sample images to a preset resolution and a preset size. Then, based on the preprocessed first sample image, a corresponding mask image is generated.

The above-mentioned preprocessing may include a first preprocessing operation and a second preprocessing operation. Exemplarily, the following steps can be used to preprocess a certain first sample image, and generate a mask image corresponding to the first sample image:

First, a first preprocessing operation is performed on the first sample image to obtain a first image with a preset resolution; then, a second preprocessing operation is performed on the first image to obtain a second image with a preset size; Finally, based on the annotation information of the first sample image for the target object and the second image, a mask image corresponding to the target object is generated.

Exemplarily, the preset resolution may be an image resolution of 1mm*1mm*1mm; specifically, a trilinear interpolation algorithm may be used to unify the first sample image into a first image with a preset resolution. The preset size may be the maximum size of all the first sample images, and the second preprocessing operation may be a padding method.

In some embodiments, the following steps can be used to train the pre-trained model:

First, invert each probability included in the probability map to obtain the target map; then, based on the target map and multiple second sample images, the goal is to restore the masked image blocks in each second sample image Model training is performed until the first cut-off condition for training is met, resulting in a pre-trained model. The above-mentioned first cut-off condition may specifically be the number of iterations, or may be the image restoration accuracy of the pre-trained model.

Since a pixel with a larger probability value in the probability map indicates that the pixel has a higher probability of being the target object, it is also the easiest to learn, while a smaller value indicates that the pixel is generally more difficult to segment. Therefore, in order to To learn the edge of the target object more accurately, you can invert each probability in the probability map, and then perform model training. The purpose of the negation operation is to change the original larger probability into a smaller probability.

Exemplarily, the above-mentioned inversion operation may specifically be that the inversion operation is not performed for the probability that the value is zero, and for the probability that the value is not zero, the value obtained after subtracting the probability from 1 is calculated, and the obtained value is used as the inversion operation. The result constitutes the probability in the target map.

Before training the pre-training model, each second sample image needs to be divided into a plurality of image blocks, and at least one image block in each second sample image needs to be masked out respectively. Exemplarily, for a certain second sample image, a plurality of image blocks formed by dividing it may be arranged into a queue, and then the order of the image blocks in the queue is shuffled, and then the queues arranged in the queue are arranged. 75% of the image blocks of the tail are masked off.

During the training of the pre-training model, the remaining image blocks of each second sample image are input into the pre-training model to be trained, and the pre-training model outputs the predicted restored image corresponding to each second sample image; The two-sample images, each predicted restoration image, and the target atlas are used to determine the image restoration loss information; finally, based on the image restoration loss information, the target is to restore the masked image blocks in each second sample image, and the pre-training The training model is trained to obtain a trained pre-trained model.

Exemplarily, when determining the above image restoration loss information, first determine the loss information of each pixel point in each masked image block based on each second sample image and each predicted restoration image, and then use the corresponding pixel point. The probability in the target map weights the loss information, which prompts the pre-training model to focus on some difficult and rare pixels, such as edge pixels, which can effectively improve the training of the pre-training model. precision. Finally, the weighted loss information of each pixel in each masked image block can be summed to obtain the above image restoration loss information.

Exemplarily, as shown in FIG. 2 , the remaining image block 2B after masking out some image blocks of a second sample image 2A or the information corresponding to the image blocks can be input into the encoder in the pre-training model, and the encoder Perform image feature processing on the input image block or information to obtain coding information 2C, and based on the obtained coding information, the information of the masked image block is inserted into the image block that has not been masked out according to its position in the image. information in a queue or list. After that, the information in the queue or list after the information insertion operation is input into the decoder in the pre-training model, and after the decoder processes the input information or features, the decoded information 2D is obtained, and the decoded information includes recovery or Of course, the information of the restored image block may also include the information of the image block that has not been masked out. Finally, based on the decoded information, a restored second sample image 2E, that is, the above-mentioned predicted restored image can be generated.

All the above masked image patches are collectively represented by a learnable vector, that is, all masked patches share this vector, so that the pre-training model knows that this position is masked.

Exemplarily, the image block or the information of the image block may be encoded by using the encoder as shown in FIG. 3 .

The training process of the above pre-training model does not require supervised information, that is, does not require labeling information. It is a process of self-supervised learning and training, which reduces the requirements for training samples. Unlabeled training samples can be used, so it is easier to obtain a large number of training samples. The training samples are beneficial to improve the training accuracy.

In some embodiments, the pre-training model is trained based on the multiple third sample images and the labeling information of the third sample images on the target object to obtain an image segmentation model for the target object. Specifically, the following steps can be used to achieve:

First, each third sample image is divided into multiple image blocks, and the image blocks corresponding to each third sample image are input into the pre-training model to obtain the predicted segmented images corresponding to each third sample image; The third sample image has the annotation information of the target object and each predicted segmentation image, and determines the image segmentation loss information; finally, based on the image segmentation loss information, the pre-training model is trained until the second cut-off condition of training is met, and the target is obtained. An image segmentation model for objects. The above-mentioned second cutoff condition may specifically be the number of iterations, or may be the segmentation accuracy of the image segmentation model.

Exemplarily, as shown in FIG. 4 , the information of the image block 4B or the image block corresponding to a certain third sample image 4A is input into the encoder in the pre-training model, and the encoder encodes the input image block or information. processing to obtain coded information 4C, and then further information processing is performed on the obtained coded information 4C to obtain processed information 4D, and then the processed information 4D is input into the decoder in the pre-training model, and the input information is processed by the decoder. Or after the feature is processed, decoded information 4E is obtained, the decoded information 4E includes segmentation information of the target object, and finally, a predicted segmented image 4F can be generated based on the decoded information 4E.

The above-mentioned image segmentation loss information includes multiple categories of image segmentation sub-loss information, such as cross-entropy sub-loss information, Dice sub-loss information, and the like. Exemplarily, the following steps can be used to train the pre-training model based on the image segmentation sub-loss information of the above-mentioned multiple categories to obtain an image segmentation model for the target object:

First, based on the sub-loss information of image segmentation of multiple categories, the target loss information is determined; finally, based on the target loss information, the pre-training model is trained to obtain the image segmentation model for the target object.

When determining the target loss information, the sum of the sub-loss information of image segmentation of multiple categories can be used as the target loss information. Of course, the target loss information can also be obtained by performing a weighted sum operation on the image segmentation sub-loss information of multiple categories.

The initial parameters of the above image segmentation model are the parameters obtained from the training of the pre-training model, that is, the use of high-quality initial parameters to train the image segmentation model can reduce the number of training samples with labeled information to be used. The accuracy of image segmentation is improved, the number of required training samples can also be reduced, and the method is suitable for the training of segmentation models for medical image segmentation. For example, the adam optimizer can be used in the specific training process, its learning rate can be 10-5, and the training round can be 200 rounds, and finally an image segmentation model for liver segmentation can be obtained.

To sum up, as shown in FIG. 5 , the above-mentioned network model training method of the present disclosure may include the following steps:

First, using a plurality of first sample images with annotation information for the target object to generate a probability atlas;

Second, use the probability map and multiple second sample images without label information to perform self-supervised learning with the goal of restoring the masked image blocks in the second sample image to obtain a pre-training model.

Third, the pre-training model is further trained by using a plurality of third sample images with labeling information for the target object to obtain a trained image segmentation model.

Among them, as shown in FIG. 6 , in the first step above, when determining the probability map, it is first necessary to generate a mask image corresponding to each first sample image, and then, according to each mask image and each first sample image The annotation information of the sample image generates a probability map.

As shown in Figure 6, in the second step above, when training the pre-training model, a self-supervised learning framework is first constructed, and then the remaining image blocks or remaining image blocks after partial image blocks are masked out of each second sample image. The information of the image block is input into the pre-training model to be trained, and the pre-training model to be trained outputs the predicted restored image; after that, the loss of each pixel in each masked image block is determined based on the predicted restored image and the second sample image After that, the probability in the above probability map is reversed and then weighted with the corresponding loss information, and finally the image restoration loss information is determined based on each weighted loss information. The image restoration loss information may be the sum of the weighted loss information.

In the above embodiment, the pre-training model is first trained on a large number of general training samples without label information, and general image features are learned, and then the migration training is carried out for the task in a targeted manner, and the training method of the pre-training model can use self-supervision. Learn techniques to do it. The trained parameters of the pre-training model are used as the initialization parameters of the image segmentation model to perform migration learning, so that a high-precision image segmentation model is finally obtained. The method in the above embodiment can be applied to the scene with less label information in the field of medical image segmentation, combined with the self-supervised learning in the transfer learning method to obtain the pre-training model, and the way of obtaining the pre-training model is optimized, and the target object is obtained. Its own characteristics and prior knowledge, that is, probability maps, are integrated into the training of the pre-training model, which prompts the pre-training model to focus on and focus on some important areas of the target object during the training process, so as to promote the use of the pre-training model for Transferring to target object segmentation can further improve the segmentation accuracy of the trained image segmentation model.

The method of the above embodiment is suitable for medical image segmentation. Based on the characteristics of objects in the field of medical image segmentation, it is proposed to apply the probability map theory in the self-supervised learning framework, and then use the pre-trained model trained by the self-supervised learning framework to be used. Transfer learning in medical image segmentation. On the one hand, the use of pre-trained models obtained from self-supervised learning can effectively improve the accuracy of medical image segmentation tasks with a small sample size. Combined, it can make the self-supervised learning framework pay more attention to the target object in image segmentation in a more targeted manner, and further improve the migration ability of the pre-training model to downstream tasks (medical image segmentation tasks).

As shown in FIG. 7 , the present disclosure also provides a training method for an image restoration model, which may specifically include the following steps:

S710. Generate a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object.

S720 , based on the probability map and the plurality of second sample images, perform model training with the goal of restoring the masked image blocks in each second sample image to obtain an image restoration model.

The above steps S710 to S720 are the same as the steps S110 to S120 in the above embodiment, and the image restoration model corresponds to the pre-training model, so the same content will not be repeated.

In some embodiments, the probability map includes the probability that each pixel in an image of a preset size belongs to the target object. The above-mentioned model training is performed based on the probability map and multiple second sample images with the goal of recovering the masked image blocks in each second sample image, and the image restoration model is obtained, which can be realized by the following steps:

First, invert the probabilities included in the probability map to obtain the target map; then, based on the target map and multiple second sample images, restore the masked image blocks in each second sample image as the goal. The model is trained to obtain an image restoration model.

Using probabilistic maps and unlabeled images for training not only reduces the requirements for training samples, but also makes it easier to obtain a large number of samples, and the combination of prior knowledge information (probability maps) and self-supervised learning can be more targeted. Self-supervised learning pays more attention to the target object and obtains an image restoration model with higher accuracy.

Since a pixel with a larger probability value in the probability map indicates that the pixel has a higher probability of being the target object, it is also the easiest to learn, while a smaller value indicates that the pixel is generally more difficult to segment. Therefore, in order to To learn the edge of the target object more accurately, each probability in the probability map can be reversed, and then model training is performed, which is beneficial to improve the accuracy of restoring or restoring the edge of the target object.

Based on the same inventive concept, the embodiments of the present disclosure also provide a network model training device corresponding to the network model training method, which is used for training an image segmentation model. For example, the above-mentioned network model training method is similar, so the implementation of the device can refer to the implementation of the method, and the repetition will not be repeated.

As shown in FIG. 8, a schematic structural diagram of a network model training apparatus provided by an embodiment of the present disclosure includes:

The first atlas determination module 810 is configured to generate a probability atlas corresponding to the target object based on the annotation information of the plurality of first sample images for the target object.

The pre-training module 820 is configured to perform model training based on the probability map and the plurality of second sample images, with the goal of restoring the masked image blocks in each second sample image, to obtain a pre-training model.

The segmentation model training module 830 is configured to train the pre-training model based on the plurality of third sample images and the labeling information of the target object by the third sample images to obtain an image segmentation model for the target object.

In some embodiments, the probability map includes the probability that each pixel in an image of a preset size belongs to the target object;

The pre-training module 820 is specifically used for:

Invert each probability included in the probability map to obtain the target map;

Based on the target atlas and multiple second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain a pre-training model.

In some embodiments, the pre-training module 820 is specifically used to:

For each second sample image, dividing the second sample image into a plurality of image blocks, and masking out at least one image block;

Input the remaining image blocks of each second sample image into the pre-training model to be trained to obtain the predicted restoration image corresponding to each second sample image;

Determine image restoration loss information based on each second sample image, each predicted restoration image, and the target atlas;

Based on the image restoration loss information, with the goal of restoring the masked image blocks in each second sample image, the pre-training model to be trained is trained to obtain a trained pre-training model.

In some embodiments, the image of the preset size has the same resolution and size as the second sample image.

In some embodiments, the first map determination module 810 is specifically configured to:

For each first sample image, a mask image of the first sample image for the target object is generated based on the annotation information of the first sample image for the target object;

Based on the mask image corresponding to each second sample image, a probability map corresponding to the target object is generated.

In some embodiments, the first map determination module 810 is specifically configured to:

performing a first preprocessing operation on the first sample image to obtain a first image with a preset resolution;

performing a second preprocessing operation on the first image to obtain a second image with a preset size;

Based on the annotation information of the first sample image for the target object and the second image, a mask image corresponding to the target object is generated.

In some embodiments, the segmentation model training module 830 is specifically used to:

Dividing each third sample image into a plurality of image blocks, and inputting the image blocks corresponding to each third sample image into the pre-training model to obtain a predicted segmented image corresponding to each third sample image;

Determine the image segmentation loss information based on the annotation information of each third sample image to the target object and each predicted segmentation image;

Based on the image segmentation loss information, the pre-trained model is trained to obtain an image segmentation model for the target object.

In some embodiments, the image segmentation loss information includes multiple categories of image segmentation sub-loss information;

The segmentation model training module 830 is specifically used for:

Determine the target loss information based on the image segmentation sub-loss information of multiple categories;

Based on the target loss information, the pre-trained model is trained to obtain an image segmentation model for the target object.

Based on the same inventive concept, the embodiment of the present disclosure also provides a network model training device corresponding to the network model training method, which is used for training an image restoration model. For example, the above-mentioned network model training method is similar, so the implementation of the device can refer to the implementation of the method, and the repetition will not be repeated.

As shown in FIG. 9, a schematic structural diagram of a network model training apparatus provided by an embodiment of the present disclosure includes:

The second atlas determination module 910 is configured to generate a probability atlas corresponding to the target object based on the annotation information of the plurality of first sample images for the target object;

The restoration model training module 920 is configured to perform model training with the goal of restoring the masked image blocks in each second sample image based on the probability map and the plurality of second sample images to obtain an image restoration model.

In some embodiments, the probability map includes the probability that each pixel in an image of a preset size belongs to the target object;

The recovery model training module 920 is specifically used for:

Invert each probability included in the probability map to obtain the target map;

Based on the target atlas and multiple second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain an image restoration model.

In the technical solution of the present disclosure, the acquisition, storage and application of the user's personal information involved are all in compliance with the provisions of relevant laws and regulations, and do not violate public order and good customs.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

10 shows a schematic block diagram of an example electronic device 1000 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 10 , the device 1000 includes a computing unit 1010 that can be executed according to a computer program stored in a read only memory (ROM) 1020 or a computer program loaded from a storage unit 1080 into a random access memory (RAM) 1030 Various appropriate actions and handling. In the RAM 1030, various programs and data necessary for the operation of the device 1000 can also be stored. The computing unit 1010 , the ROM 1020 , and the RAM 1030 are connected to each other through a bus 1040 . An input/output (I/O) interface 1050 is also connected to the bus 1040 .

Various components in the device 1000 are connected to the I/O interface 1050, including: an input unit 1060, such as a keyboard, a mouse, etc.; an output unit 1070, such as various types of displays, speakers, etc.; a storage unit 1080, such as a magnetic disk, an optical disk, etc. ; and a communication unit 1090, such as a network card, modem, wireless communication transceiver, and the like. The communication unit 1090 allows the device 1000 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 1010 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 1010 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1010 performs the various methods and processes described above, such as the method network model training method. For example, in some embodiments, the network model training method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1080 . In some embodiments, part or all of the computer program may be loaded and/or installed on the device 500 via the ROM 1020 and/or the communication unit 1090 . When the computer program is loaded into RAM 1030 and executed by computing unit 1010, one or more steps of the network model training method described above may be performed. Alternatively, in other embodiments, the computing unit 1010 may be configured to perform the network model training method by any other suitable means (eg, by means of firmware).

Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a distributed system server, or a server combined with blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (24)

1. A network model training method, comprising: generating a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object; Based on the probability map and the plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain a pre-training model; The pre-training model is trained based on the plurality of third sample images and the labeling information of the target object by the third sample images to obtain an image segmentation model for the target object. 2. The method according to claim 1, wherein the probability map comprises the probability that each pixel in the image of a preset size belongs to the target object; Based on the probability map and a plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain a pre-training model, including: Perform an inversion operation on each probability included in the probability map to obtain a target map; Based on the target atlas and the plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain a pre-trained model. 3. The method according to claim 2, wherein, based on the target atlas and a plurality of second sample images, the model training is performed with the goal of restoring the masked image blocks in each second sample image, Get a pretrained model, including: For each second sample image, dividing the second sample image into a plurality of image blocks, and masking out at least one image block; Input the remaining image blocks of each second sample image into the pre-training model to be trained to obtain the predicted restoration image corresponding to each second sample image; Determine image restoration loss information based on each second sample image, each predicted restoration image, and the target atlas; Based on the image restoration loss information, the pre-training model to be trained is trained with the goal of restoring the masked image blocks in each of the second sample images to obtain a trained pre-training model. 4. The method according to claim 2 or 3, wherein the image of the preset size has the same resolution and size as the second sample image. 5. The method according to any one of claims 1 to 4, wherein the generating a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object, comprising: For each first sample image, based on the annotation information of the first sample image for the target object, generate a mask image of the first sample image for the target object; Based on the mask images corresponding to each of the first sample images, a probability map corresponding to the target object is generated. 6. The method according to claim 5, wherein generating a mask image of the first sample image for the target object based on the annotation information of the first sample image for the target object comprises: performing a first preprocessing operation on the first sample image to obtain a first image with a preset resolution; performing a second preprocessing operation on the first image to obtain a second image with a preset size; Based on the annotation information of the first sample image for the target object and the second image, a mask image corresponding to the target object is generated. 7. The method according to any one of claims 1 to 6, wherein the pre-training model is trained based on the labeling information of the target object based on the plurality of third sample images and the third sample images to obtain the target object. The image segmentation model of the target object includes: Dividing each of the third sample images into a plurality of image blocks, and inputting the image blocks of each third sample image into a pre-training model to obtain a predicted segmented image corresponding to each third sample image; Determine image segmentation loss information based on the annotation information of each third sample image to the target object and each predicted segmentation image; Based on the image segmentation loss information, the pre-trained model is trained to obtain an image segmentation model for the target object. 8. The method of claim 7, wherein the image segmentation loss information comprises image segmentation sub-loss information of multiple categories; The pre-training model is trained based on the image segmentation loss information to obtain an image segmentation model for the target object, including: Determine target loss information based on the image segmentation sub-loss information of the multiple categories; Based on the target loss information, the pre-training model is trained to obtain an image segmentation model for the target object. 9. The method of any one of claims 1 to 8, wherein the first sample image comprises a medical image; and/or, the second sample image includes a medical image; and/or, The third sample image includes a medical image. 10. A network model training method, comprising: generating a probability map corresponding to the target object based on the annotation information of the plurality of first sample images for the target object; Based on the probability map and the plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain an image restoration model. 11. The method according to claim 10, wherein the probability map comprises the probability that each pixel in the image of preset size belongs to the target object; Based on the probability map and multiple second sample images, model training is performed with the goal of recovering the masked image blocks in each second sample image, and an image restoration model is obtained, including: Perform an inversion operation on each probability included in the probability map to obtain a target map; Based on the target atlas and the plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain an image restoration model. 12. A network model training device, comprising: a first atlas determination module, configured to generate a probability atlas corresponding to the target object based on the annotation information of the plurality of first sample images for the target object; A pre-training module for performing model training based on the probability atlas and a plurality of second sample images with the goal of restoring the masked image blocks in each second sample image to obtain a pre-training model; The segmentation model training module is used for training the pre-training model based on the plurality of third sample images and the labeling information of the target object by the third sample images to obtain an image segmentation model for the target object. 13. The apparatus according to claim 12, wherein the probability map comprises the probability that each pixel in the image of a preset size belongs to the target object; The pre-training module is specifically used for: Perform an inversion operation on each probability included in the probability map to obtain a target map; Based on the target atlas and the plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain a pre-trained model. 14. The apparatus according to claim 13, wherein the pre-training module is specifically used for: For each second sample image, dividing the second sample image into a plurality of image blocks, and masking out at least one image block; Input the remaining image blocks of each second sample image into the pre-training model to be trained to obtain the predicted restoration image corresponding to each second sample image; Determine image restoration loss information based on each second sample image, each predicted restoration image, and the target atlas; Based on the image restoration loss information, the pre-training model to be trained is trained with the goal of restoring the masked image blocks in each of the second sample images to obtain a trained pre-training model. 15. The apparatus of claim 13 or 14, wherein the image of the preset size has the same resolution and size as the second sample image. 16. The device according to any one of claims 12 to 15, wherein the first map determination module is specifically used for: For each first sample image, based on the annotation information of the first sample image for the target object, generate a mask image of the first sample image for the target object; Based on the mask image corresponding to each second sample image, a probability map corresponding to the target object is generated. 17. The apparatus according to claim 16, wherein the first map determination module is specifically used for: performing a first preprocessing operation on the first sample image to obtain a first image with a preset resolution; performing a second preprocessing operation on the first image to obtain a second image with a preset size; Based on the annotation information of the first sample image for the target object and the second image, a mask image corresponding to the target object is generated. 18. The apparatus according to any one of claims 12 to 17, wherein the segmentation model training module is specifically used for: Divide each of the third sample images into a plurality of image blocks, and input the image blocks corresponding to each of the third sample images into the pre-training model to obtain predicted segmentation images corresponding to each of the third sample images; Determine image segmentation loss information based on the annotation information of each third sample image to the target object and each predicted segmentation image; Based on the image segmentation loss information, the pre-trained model is trained to obtain an image segmentation model for the target object. 19. The apparatus of claim 18, wherein the image segmentation loss information comprises image segmentation sub-loss information of a plurality of categories; The segmentation model training module is specifically used for: Determine target loss information based on the image segmentation sub-loss information of the multiple categories; Based on the target loss information, the pre-trained model is trained to obtain an image segmentation model for the target object. 20. A network model training device, comprising: a second atlas determination module, configured to generate a probability atlas corresponding to the target object based on the annotation information of the plurality of first sample images for the target object; The restoration model training module is configured to perform model training with the goal of restoring the masked image blocks in each second sample image based on the probability map and the plurality of second sample images to obtain an image restoration model. 21. The apparatus according to claim 20, wherein the probability map comprises the probability that each pixel in the image of a preset size belongs to the target object; The recovery model training module is specifically used for: Perform an inversion operation on each probability included in the probability map to obtain a target map; Based on the target atlas and the plurality of second sample images, model training is performed with the goal of restoring the masked image blocks in each second sample image to obtain an image restoration model. 22. An electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any one of claims 1 to 11 Methods. 23. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1 to 11. 24. A computer program product comprising a computer program/instructions, wherein the computer program/instructions, when executed by a processor, implement the method of any one of claims 1 to 11.

Images