WO2023030427A1

WO2023030427A1 - Training method for generative model, polyp identification method and apparatus, medium, and device

Info

Publication number: WO2023030427A1
Application number: PCT/CN2022/116426
Authority: WO
Inventors: 边成; 石小周; 杨延展
Original assignee: 北京字节跳动网络技术有限公司
Priority date: 2021-09-02
Filing date: 2022-09-01
Publication date: 2023-03-09
Also published as: CN113469295B; CN113469295A

Abstract

A training method for a generative model, a polyp identification method and apparatus, a medium, and a device, the method comprising: acquiring a training sample set, each training sample in the training sample set comprising a training image and a polyp labeling category corresponding to the training image; according to the training image and an image generation model, obtaining a generated image and a restored image corresponding to the training image; according to the training image and the generated image, determining a first distribution distance corresponding to the training image and the generated image; determining target loss of the image generation model according to the first distribution distance, the training image, the generated image, the restored image and a polyp labeling category corresponding to the training image, wherein the target loss comprises first distribution loss determined according to the first distribution distance, and the first distribution loss and the first distribution distance have a negative correlation relationship; and insofar as an updating condition is met, updating parameters of the image generation model according to the target loss.

Description

Training method for generative model, polyp identification method, device, medium and equipment

Cross References to Related Applications

This application is based on a Chinese patent application with the application number 202111028344.X and the filing date of September 02, 2021, entitled "Generative model training method, polyp identification method, device, medium and equipment", and requires the Chinese patent The priority of the application, the entire content of the Chinese patent application is hereby incorporated into this application as a reference.

technical field

The present disclosure relates to the field of image processing, and in particular, relates to a training method for generating a model, a method for identifying polyps, a device, a medium, and equipment.

Background technique

Endoscopes are widely used for colon screening and polyp detection, but the detection accuracy of endoscopes largely depends on the experience of endoscopists. However, because the characteristics of polyps are difficult to identify, and many polyps are small in size, the missed detection rate of polyp detection is relatively high, which greatly increases the difficulty of early polyp screening.

In related technologies, the deep learning method can be used for model training to be used in a computer-aided diagnosis system for polyp identification and segmentation. However, when the out-of-sample data has a large domain shift through the above method, there may be a large performance gap in these trained networks, and it is difficult to ensure the generalization of the model through limited sample data, making the trained model The detection accuracy of out-of-sample data is insufficient to achieve accurate polyp detection.

Contents of the invention

This Summary is provided to introduce a simplified form of concepts that are described in detail later in the Detailed Description. This summary of the invention is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

In a first aspect, the present disclosure provides a method for training a polyp image generation model, the method comprising:

Obtaining a training sample set, wherein each training sample in the training sample set includes a training image and a polyp label category corresponding to the training image;

According to the training image and the image generation model, a generated image and a restored image corresponding to the training image are obtained, wherein the image generation model includes a first generator and a second generator, and the first generator is used to The training image is used to generate the generated image, and the second generator is used to generate the restored image according to the generated image;

According to the training image and the generated image, determine a first distribution distance corresponding to the training image and the generated image, wherein the first distribution distance is used to represent the distribution of the training image and the generated image The difference between the distributions of ;

According to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, determine the target loss of the image generation model, wherein the target loss includes The first distribution loss determined according to the first distribution distance, the first distribution loss and the first distribution distance are negatively correlated;

If the updating condition is met, the parameters of the image generation model are updated according to the target loss.

In a second aspect, the present disclosure provides a polyp identification method, the method comprising:

receiving the polyp image to be identified;

Inputting the polyp image into a polyp recognition model to obtain a recognition result of the polyp image, wherein the training sample set corresponding to the polyp recognition model includes an original sample, and according to the original sample and the first generated image in the image generation model The generation sample generated by the machine, the image generation model is obtained by training based on the polyp image generation model training method described in the first aspect, the original sample includes the original image and the polyp label category corresponding to the original image, so The generated samples include a generated image generated based on an original image and a polyp labeling category corresponding to the original image.

In a third aspect, the present disclosure provides a training device for a polyp image generation model, the device comprising:

An acquisition module, configured to acquire a training sample set, wherein each training sample in the training sample set includes a training image and a polyp label category corresponding to the training image;

A generation module, configured to obtain a generated image and a restored image corresponding to the training image according to the training image and the image generation model, wherein the image generation model includes a first generator and a second generator, and the first The generator is used to generate the generated image according to the training image, and the second generator is used to generate the restored image according to the generated image;

A first determining module, configured to determine a first distribution distance corresponding to the training image and the generated image according to the training image and the generated image, wherein the first distribution distance is used to represent the training image The difference between the distribution of and the distribution of the generated images;

The second determination module is configured to determine the target loss of the image generation model according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, Wherein, the target loss includes a first distribution loss determined according to the first distribution distance, and the first distribution loss is negatively correlated with the first distribution distance;

An update module, configured to update the parameters of the image generation model according to the target loss when an update condition is met.

In a fourth aspect, a polyp identification device is provided, the device comprising:

A receiving module, configured to receive an image of a polyp to be identified;

An identification module, configured to input the polyp image into a polyp identification model to obtain an identification result of the polyp image, wherein the training sample set corresponding to the polyp identification model includes original samples, and a model is generated according to the original samples and images The generation sample generated by the first generator in the first aspect, the image generation model is obtained by training based on the training method of the polyp image generation model described in the first aspect, and the original sample includes the original image and the corresponding The polyp labeling category, the generated samples include a generated image generated based on the original image and a polyp labeling category corresponding to the original image.

In a fifth aspect, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processing device, the steps of the method described in the first or second aspect are implemented.

In a sixth aspect, an electronic device is provided, including:

a storage device on which a computer program is stored;

A processing device configured to execute the computer program in the storage device to implement the steps of the method in the first or second aspect.

Through the above technical solution, a new image can be generated based on the training image and the image generation model, and the generated image and the restored image can be obtained. When determining the target loss of the image generation model, through the constraints of the restored image and the polyp labeling category, it can be generated based on confrontation The style transfer method of the network imitates and generates the training images to ensure the semantic consistency between the generated images generated based on the image generation model and the original images, so that the generated images generated by the image generation model and the training images belong to the same polyp classification, Furthermore, there is no need to perform data labeling on the generated images, and effectively labeled samples for polyp recognition model training can be automatically generated. And when determining the target loss, the first distribution distance between the training image and the generated image is also determined, then the first distribution distance can be further combined on the basis of combining the training sample, the generated image and the restored image to determine the target loss for image generation models. Therefore, through the first distribution loss, more data with diversity can be obtained on the basis of not obtaining additional polyp categories in the generated image obtained based on the image generation model, so that it can be guaranteed that based on the generated image and the training image The generalization of the model being trained, such as the polyp recognition model. The polyp image is generated by the image generation model, so that more training data for training the polyp recognition model can be obtained based on limited sample data, which can reduce the manpower and time spent on polyp recognition model training, and can further improve polyp recognition The detection accuracy and robustness of the model ensure the accuracy of polyp detection and effectively reduce the missed detection rate of polyp detection.

Other features and advantages of the present disclosure will be described in detail in the detailed description that follows.

Description of drawings

The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale. In the attached picture:

FIG. 1 is a flowchart of a training method for a polyp image generation model provided according to an embodiment of the present disclosure;

Fig. 2 is a schematic diagram of an image generation model provided according to an embodiment of the present disclosure;

Fig. 3 is a block diagram of a training device for a polyp image generation model provided according to an embodiment of the present disclosure;

FIG. 4 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein; A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the various steps described in the method implementations of the present disclosure may be executed in different orders, and/or executed in parallel. Additionally, method embodiments may include additional steps and/or omit performing illustrated steps. The scope of the present disclosure is not limited in this respect.

As used herein, the term "comprise" and its variations are open-ended, ie "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one further embodiment"; the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the sequence of functions performed by these devices, modules or units or interdependence.

It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more" multiple".

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only, and are not used to limit the scope of these messages or information.

As shown in the background art, it has been proposed in the related art to apply a convolutional neural network-based deep learning model to the automatic detection and identification of polyps. However, in the use of the model in this way, the problem of performance degradation caused by low generalization ability is often encountered, and it is difficult to guarantee the accuracy of polyp detection. Based on this, the present disclosure provides the following embodiments, by training the image generation model to generate more diverse training data based on the existing training data, thereby improving the generalization and detection accuracy of the polyp recognition model trained based on the training data Spend.

As shown in FIG. 1, it is a flowchart of a training method for a polyp image generation model provided according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes:

In step 11, a training sample set is acquired, wherein each training sample in the training sample set includes a training image and a labeled category of polyps corresponding to the training image.

Illustratively, endoscopic images (such as gastroscopic images, colonoscopic images, etc.) of multiple patients including polyps can be collected as training images under real conditions. As an example, data collection can be performed on patients to obtain detection data containing polyps, and then in order to ensure uniform processing of training images, the detection data can be standardized, for example, the obtained detection data contains white light endoscopic images of polyps as the training image. Further, the resolution and size of the training image can be standardized to obtain a training image of uniform size, which facilitates the subsequent training process. For each training image, an experienced gastrointestinal endoscopist may mark the corresponding polyp label, that is, the polyp label category.

In step 12, according to the training image and the image generation model, the generated image and restored image corresponding to the training image are obtained, wherein the image generation model includes a first generator and a second generator, and the first generator is used to The training image generates the generated image, and the second generator is configured to generate the restoration image according to the generated image.

Wherein, the image generation model can be realized based on the CycleGAN network, including two generators in the CycleGAN network, as shown in Figure 2, the image generation model can include a first generator 21 and a second generator 22, as shown in Figure 2 As shown, the training image can be input to the first generator 21, so that the first generator 21 can generate the corresponding generated image X _k _{' and the corresponding polyp of the generated image according to the training image X k} and the polyp labeling category k corresponding to the training image. Category k', wherein, in the process of image generation, based on the style transfer method of the confrontation generation network, the polyp label category in the training sample set is used as a condition to generate a new generated image based on the original image, such as:

X _k' = G(X _k ,k')

Wherein, X _k' is used to represent the generated image, k' is used to represent the polyp category corresponding to the generated image, and the polyp label category k corresponding to the training image X _k can be directly used as the polyp category corresponding to the generated image, G (,) is used to represent the image generation operation of the first generator. As shown in Figure 2, the image generation model contains two generators, which constitute a ring network, that is _, the second generator can further generate a training image X k corresponding to the generated image X _k' and the polyp category corresponding to the generated image The restored image X′ _k' :

X'_k' = F(G(X _k ,k'),k);

Wherein, F(,) is used to represent the image generation operation of the second generator. Therefore, through the above steps, more diverse training images can be generated based on the image generation model without adding additional types of polyps.

In step 13, according to the training image and the generated image, the first distribution distance corresponding to the training image and the generated image is determined, wherein the first distribution distance is used to represent the difference between the distribution of the training image and the distribution of the generated image difference between.

Among them, in the embodiment of the present disclosure, the purpose of generating a new image based on the training image is to generate more diverse data corresponding to the training image. Therefore, in this step, two parameters can be determined based on the distribution of the training image and the distribution of the generated image. distribution distance between them. When it needs to be explained, the generated image itself is generated based on the training image and the polyp label category of the training image as a constraint condition. Therefore, the generated image and the polyp category corresponding to the training image are the same category. In this embodiment , the distribution distance between the training image and the generated image can be increased to make the distribution of the newly generated image different from the training image, so as to ensure the diversity of the new generated image.

In step 14, the target loss of the image generation model is determined according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, wherein the target loss includes The determined first distribution loss has a negative correlation with the first distribution distance, and the larger the first distribution distance is, the smaller the first distribution loss is.

In step 15, if the update condition is satisfied, the parameters of the image generation model are updated according to the target loss.

As an example, the update condition may be that the target loss is greater than a preset loss threshold, which means that the accuracy of the image generation model is insufficient. As another example, the update condition may be that the number of iterations is less than a preset number threshold, at this time, it is considered that the number of iterations of the image generation model is small, and its accuracy is insufficient.

Correspondingly, when the update condition is met, the parameters of the image generation model can be updated according to the target loss. Wherein, the manner of updating the parameters based on the determined target loss may adopt an updating manner commonly used in the art, so that the target loss can gradually converge, and will not be repeated here.

If the update condition is not met, it can be considered that the accuracy of the image generation model meets the training requirements, and at this time the training process can be stopped to obtain a trained image generation model.

Thus, through the above technical solution, a new image can be generated based on the training image and the image generation model, and the generated image and the restored image can be obtained. When determining the target loss of the image generation model, through the constraints of the restored image and the polyp labeling category, it can be The style transfer method based on the confrontation generation network imitates and generates the training image to ensure the semantic consistency between the generated image generated based on the image generation model and the original image, so that the generated image generated by the image generation model and the training image belong to the same Polyp classification, so that there is no need to label the generated images, and effective labeled samples for polyp recognition model training can be automatically generated. And when determining the target loss, according to the training image and the generated image, determine the first distribution distance between the two, then you can further combine the first distribution distance on the basis of combining the training sample, the generated image and the restored image to determine the target loss for image generation models. Therefore, through the first distribution loss, more data with diversity can be obtained on the basis of not obtaining additional polyp categories in the generated image obtained based on the image generation model, so that it can be guaranteed that based on the generated image and the training image The generalization of the model being trained, such as the polyp recognition model. The polyp image is generated by the image generation model, so that more training data for training the polyp recognition model can be obtained based on limited sample data, which can reduce the manpower and time spent on polyp recognition model training, and can further improve polyp recognition The detection accuracy and robustness of the model ensure the accuracy of polyp detection and effectively reduce the missed detection rate of polyp detection.

In order to make those skilled in the art better understand the training method of the polyp detection model provided by the present disclosure, the above-mentioned steps are described in detail below.

In a possible embodiment, in step 13, according to the training image and the generated image, an exemplary implementation manner of determining the first distribution distance corresponding to the training image and the generated image is as follows, and this step may include:

For the training images and generated images under the same polyp labeling category, the transmission distance between the training images, the transmission distance between the generated images, and the transmission distance between the training images and the generated images are determined.

Wherein, the transmission distance can be used to measure the distance between two distributions, specifically, the transmission distance is determined by the following formula:

Wherein, W _c (X ₁ , X ₂ ) is used to represent the transmission distance between image X ₁ and image X ₂ ;

φ(X ₁ ) is used to represent the feature image extracted from the image X ₁ ;

φ(X ₂ ) is used to represent the feature image extracted from the image X ₂ ;

P ₁ is used to represent the distribution corresponding to the image X ₁ ; P ₂ is used to represent the distribution corresponding to the image X ₂ ;

∏(P ₁ , P ₂ ) is used to represent all joint distributions formed by distribution P ₁ and distribution P ₂ ;

c(X ₁ , X ₂ ) is used to represent the transmission cost between the image X ₁ and the image X ₂ .

Correspondingly, when calculating the transmission distance between training images, image X ₁ and image X ₂ are two training images sampled from the training image, and when calculating the transmission distance between generated images, image X ₁ and image X ₂ is two generated images sampled from the generated image. When calculating the transmission distance between the training image and the generated image, the image X ₁ and the image X ₂ are respectively collected from the training image and the generated image. For example, image X ₁ is a training image, and image X ₂ is a generated image.

Hereinafter, the calculation of the transmission distance between the training image and the generated image will be described in detail as an example.

Among them, all the joint distributions formed by the distribution _P1 of the training image and the distribution _P2 of the generated image can be determined first. For each possible joint distribution π, one sample image X ₁ and one sample image X ₂ can be obtained by sampling X ₁ , X ₂ ~π, and the transportation cost c(X ₁ ,X ₂ ). Wherein, in the embodiment of the present disclosure, feature extraction can be performed on the image based on CNN (Convolutional Neural Networks, convolutional neural network), that is, the feature extraction can be performed on the training image _X1 through CNN, and the corresponding feature corresponding to the training image _X1 can be obtained For the image φ(X ₁ ), feature extraction is performed on the generated image X ₂ through CNN to obtain the feature image φ(X ₂ ) corresponding to the generated image X ₂ . Afterwards, the above formula can be used to calculate based on the extracted feature image to obtain the corresponding transportation cost. Wherein, |||| ₂ is used to represent calculation in the second normal form, and the calculation method in the second normal form is the prior art, and will not be repeated here. After the transportation cost is calculated, the expected value of the sample image for the transportation cost under the joint distribution π can be calculated

A lower bound on the expected value under all possible joint distributions

That is the transmission distance.

Correspondingly, the calculation methods for the transmission distance between the training images and the transmission distance between the generated images are similar to the above, and will not be repeated here.

Afterwards, the first distribution distance may be determined according to the transmission distance between the training image and the generated image, the transmission distance between the training images, and the transmission distance between the generated images.

For example, it can be calculated by the following formula:

d(P ₁ ,P ₂ )=2E[W _c (X ₁ ,X ₂ )]-E[W _c (X ₁ ,X′ ₁ )]-E[W _c (X ₂ ,X′ ₂ )]

Among them, d(P ₁ , P ₂ ) is used to represent the first distribution distance between the distribution P ₁ corresponding to the training image and the second distribution P ₂ corresponding to the generated image; for example, X ₁ and X′ ₁ can be used for Represents two sample images in the training image; X ₂ and X' ₂ can be used to represent two sample images in the generated image, so that the first distribution distance can be further determined.

Therefore, through the above technical solution, the transmission cost between the images can be calculated to further determine the transmission distance between the training image and the generated image based on the transmission cost, so as to characterize the difference between the distribution of the training image and the generated image , so as to facilitate the adjustment in the direction of increasing the difference when adjusting the model parameters in the future, and provide data support to ensure the difference between the distribution of the training image and the generated image, thereby effectively ensuring that the image generated based on the trained image generation model Variety of generated images.

In a possible embodiment, in step 14, according to the first distribution distance, the training image, the generated image, the restored image, and the polyp label category corresponding to the training image, an exemplary implementation of determining the target loss of the image generation model is as follows , this step can include:

A generation loss of the image generation model is determined according to the training image and the restored image corresponding to the training image.

It can be seen from the above that the generated images in the present disclosure are generated based on the training images through style transfer, which can generate generated images with diversity. In this embodiment, in order to further ensure the accuracy of the semantic information of the image, the first normal form calculation is performed according to the training image and the restoration image corresponding to the training image to obtain the generation loss, wherein the restoration image corresponding to the training image is based on The generated image corresponding to the training image is generated. Exemplarily, the generation loss L _cycle can be expressed as:

L _cycle ＝||F(G(X _k ,k'),k)-X _k || ₁

Therefore, in the embodiments of the present disclosure, the semantic consistency between the generated image and the training image can be ensured by calculating the difference between the training image and the restoration image.

Based on the polyp label category corresponding to the training image and the polyp prediction category corresponding to the generated image generated based on the training image, the prediction loss of the image generation model is determined.

For example, the generated image may be input into a discriminator corresponding to the first generator, so that a polyp prediction category corresponding to the generated image may be obtained. In the present disclosure, when the first generator generates a new image, it is constrained by the polyp labeling category of the training image, so the generated image generated by the first generator should belong to the same polyp labeling category as the training image. Therefore, the difference between the polyp labeling category corresponding to the training image and the polyp prediction category corresponding to the generated image can be calculated to ensure that the newly generated generated image and the original image belong to the same category of images, which can be expanded through the method of style transfer. At the same time, the diversity of the data set is enhanced, and the polyp category of the generated image can be automatically marked to further ensure the semantic consistency between the generated image and the training image.

For example, the prediction loss may be calculated as the cross entropy between the polyp labeled category and the polyp predicted category, and the calculation method of the cross entropy is the prior art, which will not be repeated here.

It should be noted that the parameters of the discriminator corresponding to the first generator can be adjusted synchronously with the parameters of the first generator. For example, the negative value of the prediction loss can be used as the loss of the discriminator, so as to adjust the parameters of the discriminator according to the loss, so that the accuracy of the discriminator can be improved, and the first generator can be further improved by confrontation generation. image generation accuracy.

determining a negative value of the first distribution distance as the first distribution loss;

The target loss is determined based on the generation loss, the prediction loss, and the first distribution loss.

Exemplarily, a weighted sum of generation loss, the prediction loss and the first distribution loss may be determined as the target loss. For example, the weights respectively corresponding to the generation loss, the prediction loss and the first distribution loss may be set according to specific application scenarios, which is not limited in the present disclosure.

Therefore, through the above technical solution, when determining the target loss of the image generation model, the difference between the training image and the restoration image can be calculated to ensure the semantic consistency between the generated image and the training image, and further Considering the difference between the polyp prediction category corresponding to the generated image and the polyp labeling category corresponding to the training image, the accuracy of the semantic information of the generated image is further ensured, and reliable data support is provided for automatically labeling the polyp category on the generated image. At the same time, the first distribution loss can also be combined in the target loss, so that the image generation model obtained by training can generate a variety of generated images, while ensuring the semantic consistency between the generated image and the training image, ensuring certainty the reliability of the generated samples.

In a possible embodiment, the training sample set includes training samples corresponding to multiple labeled categories of polyps, so that generated images of multiple categories can be generated based on the trained image generation model.

Correspondingly, according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, an example of determining the target loss of the image generation model The implementation is as follows, and on the basis of the example described above, this step may also include:

According to the generated images under various polyp labeling categories, for the generated images under any two polyp labeling categories, determine the second distribution distance corresponding to the generated images under the two polyp labeling categories, wherein the second distribution distance is used to represent the difference between distributions of generated images belonging to different polyp annotation categories.

Among them, in the embodiment of the present disclosure, the image generation model can be trained by training samples under various categories. In order to further ensure that the generated data has greater diversity and the images under various categories can be accurately distinguished, this In the disclosure, the difference between generated images of different categories can be ensured by determining the second distribution distance.

For example, for each polyp label classification in the training sample set, the generated images under the two polyp label categories can be arbitrarily selected to calculate the distribution distance, wherein, the calculation method of the second distribution distance between the generated images of different categories is the same as The above calculation method for determining the first distribution distance between the training image and the generated image is the same, and will not be repeated here.

Afterwards, an exemplary implementation manner of determining the target loss according to the generation loss, the prediction loss and the distribution loss is as follows, and this step may include:

Determine the second distribution difference of the image generation model according to the second distribution distance, wherein the determined negative value of the sum of the second distribution distances between generated images in different categories can be determined as the image generation model The difference in the second distribution of .

A weighted sum of the generation loss, the prediction loss, the first distribution loss, and the second distribution loss is determined as the target loss. Wherein, in this embodiment, the first distribution loss can be the negative value of the sum of the determined first distribution distances between the training image and the generated image under each category, so that the first distribution loss can be used to represent Differences between generated images and training images under the multiple polyp labeling categories.

Therefore, through the above-mentioned technical solution, it is possible to train an image generation model suitable for image extension generation of training images under multiple polyp categories, and by ensuring the difference between the distributions of images under different polyp categories, the image generation model can be guaranteed The adaptability and accuracy of each polyp classification can effectively ensure the accuracy of the generated image generated based on the image generation model after training, and provide more diverse and accurate data support for the subsequent training of the polyp recognition model .

The present disclosure also provides a method of polyp identification, the method comprising:

An image of a polyp to be identified is received, and the image of the polyp may be an image including a polyp obtained during detection.

Inputting the polyp image into a polyp recognition model to obtain a recognition result of the polyp image, wherein the training sample set corresponding to the polyp recognition model includes an original sample, and according to the original sample and the first generated image in the image generation model The generated sample generated by the machine, the image generated model is obtained by training based on any of the above polyp image generated model training methods, the original sample includes the original image and the corresponding polyp labeling category of the original image, The generated samples include a generated image generated based on the original image and a polyp labeling category corresponding to the original image.

Thus, in this embodiment, when training the polyp recognition model, on the basis of the original training samples, the image generation model trained according to any of the above-mentioned polyp image generation model training methods can perform image Generated, so that more accurate generated samples can be obtained based on the original samples, which can effectively expand the training sample set used for polyp recognition model training, thereby improving the accuracy and efficiency of the polyp recognition model obtained by draft training, and can effectively Improve the generalization and robustness of the polyp recognition model, effectively reduce the missed detection rate of polyp recognition, and improve the accuracy of polyp recognition to a certain extent.

In a possible embodiment, the polyp recognition model is trained in the following manner:

Preprocessing the target training image in the training sample set to obtain a processed image, wherein the preprocessing includes nonlinear transformation and/or local pixel shuffling, and the target training image includes the original image and the generated image.

For example, generally relative intensity values in medical images can be used to express relevant information about imaged structures and organs. Therefore, intensity information can be used as pixel-level supervisory information. In order to preserve the relative intensity of structures in image transformation, a smooth and monotonous transformation function Bezier curve can be used for nonlinear changes. In this transformation method, a unique value can be matched for each pixel in the image to ensure a one-to-one mapping relationship in the nonlinear transformation. For example, the transformation can be performed as follows:

B(t)＝(1-t) ³ p ₀ +3(1-t) ² tp ₁ +3(1-t)t ² p ₂ +t ³ p ₃ ,t∈[0,1]

Among them, B(t) is used to represent the conversion value of the conversion function, p ₀ and p ₃ are two predefined nodes, p ₁ and p ₂ are two predefined control points, t is the fraction of the extension line length The value can be set according to the actual application scenario, which is not limited in the present disclosure. The nonlinear transformation processing of the target training image can be realized through the above method.

As another example, a window may be randomly selected from the target training image, and then the sequence of pixels in the window may be disturbed, so that a processed image corresponding to the target training image may be obtained. For example, the size of the window can be set to be smaller than the size of the corresponding receptive field in the polyp identification model.

Among them, the target training image can be preprocessed by any of the above methods to obtain the processed image, and the two methods can also be combined for preprocessing. For example, the target training image can be nonlinearly converted first and then local pixel shuffling is performed to obtain the processed image. Or the target training image can be subjected to local pixel shuffling and nonlinear transformation to obtain the processed image.

The polyp recognition model is pre-trained by using the processed image as a model input and using the target training image as a target output to obtain a pre-trained polyp recognition model.

In this step, the processed image can be used as input, so that the image recovered by the polyp recognition model can be used for loss calculation with the target training image, and the polyp recognition model can be pre-trained based on the calculated loss. When the loss is less than the threshold Or when the number of iterations meets a certain number of times, the pre-training process is ended to obtain a pre-trained polyp recognition model.

The target training image is used as a model input, and the polyp labeling category corresponding to the target training image is used as a target output to train the pre-trained polyp recognition model to obtain a trained polyp recognition model.

In this step, the target training image can be used as input, so that the predicted category output by the polyp recognition model and the polyp label category corresponding to the target training image can be used for loss calculation, and the polyp recognition model can be trained based on the calculated loss. When the loss is less than the threshold or the number of iterations meets a certain number of times, the training process is ended to obtain a trained polyp recognition model.

Thus, through the above technical solution, when the polyp recognition model is trained based on the training sample data, the training image can be preprocessed first, and the preprocessed image can be restored by the polyp recognition model as the training task, Pre-training the polyp recognition model can improve the feature learning ability in the polyp recognition model and improve the adaptability to subsequent model training tasks. Afterwards, the pre-trained polyp recognition model is trained based on the training sample set to obtain the polyp recognition model, so that the application scenarios of the polyp recognition model can be effectively broadened, and the accuracy and applicability of the polyp recognition model can be improved at the same time.

The present disclosure also provides a training device for a polyp image generation model, as shown in FIG. 3 , the device 40 includes:

An acquisition module 41, configured to acquire a training sample set, wherein each training sample in the training sample set includes a training image and a polyp label category corresponding to the training image;

The generation module 42 is configured to obtain a generated image and a restored image corresponding to the training image according to the training image and the image generation model, wherein the image generation model includes a first generator and a second generator, and the first A generator is used to generate the generated image according to the training image, and the second generator is used to generate the restored image according to the generated image;

The first determining module 43 is configured to determine a first distribution distance corresponding to the training image and the generated image according to the training image and the generated image, wherein the first distribution distance is used to represent the training the difference between the distribution of images and the distribution of said generated images;

The second determination module 44 is configured to determine the target loss of the image generation model according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image , wherein the target loss includes a first distribution loss determined according to the first distribution distance, and the first distribution loss is negatively correlated with the first distribution distance;

The update module 45 is configured to update the parameters of the image generation model according to the target loss when an update condition is met.

Optionally, the second determination module includes:

A first determination submodule, configured to determine the generation loss of the image generation model according to the training image and the restored image corresponding to the training image;

The second determining submodule is used to determine the prediction loss of the image generation model based on the polyp labeling category corresponding to the training image and the polyp prediction category corresponding to the generated image generated based on the training image;

A third determining submodule, configured to determine a negative value of the first distribution distance as the first distribution loss;

A fourth determination submodule, configured to determine the target loss according to the generation loss, the prediction loss and the first distribution loss.

Optionally, the training sample set includes training samples corresponding to multiple labeled categories of polyps;

The second determination module also includes:

The fifth determining sub-module is used to determine the second distribution distance corresponding to the generated images under the two polyp labeling categories for the generated images under any two polyp labeling categories according to the generated images under various polyp labeling categories, wherein , the second distribution distance is used to represent the difference between the distributions of generated images belonging to different polyp annotation categories;

The fourth determining submodule includes:

A sixth determining submodule, configured to determine a second distribution difference of the image generation model according to the second distribution distance;

A seventh determination submodule, configured to determine a weighted sum of the generation loss, the prediction loss, the first distribution loss, and the second distribution loss as the target loss.

Optionally, the first determination module includes:

The eighth determining submodule is used to determine the transmission distance between the training images, the transmission distance between the generated images, and the training images and the generated images for the training images and generated images under the same polyp labeling category. The transmission distance between generated images;

A ninth determining submodule, configured to determine the first distribution according to the transmission distance between the training image and the generated image, the transmission distance between the training images, and the transmission distance between the generated images distance.

Optionally, the transmission distance is determined by the following formula:

The present disclosure also provides a polyp identification device, the device comprising:

A receiving module, configured to receive an image of a polyp to be identified;

An identification module, configured to input the polyp image into a polyp identification model to obtain an identification result of the polyp image, wherein the training sample set corresponding to the polyp identification model includes original samples, and a model is generated according to the original samples and images The generation sample generated by the first generator in the above, the image generation model is obtained by training based on any of the above polyp image generation model training methods, and the original sample includes the original image corresponding to the original image polyp labeling category, the generated samples include a generated image generated based on the original image and a polyp labeling category corresponding to the original image.

Optionally, the polyp recognition model is trained in the following manner:

Preprocessing the target training image in the training sample set to obtain a processed image, wherein the preprocessing includes nonlinear transformation and/or local pixel shuffling, and the target training image includes the original image and the generated image;

Pre-training the polyp recognition model by using the processed image as a model input and using the target training image as a target output to obtain a pre-trained polyp recognition model;

The target training image is used as a model input, and the polyp label category corresponding to the target training image is used as a target output to train the pre-trained polyp recognition model to obtain a trained polyp recognition model.

Referring now to FIG. 4 , it shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal equipment in the embodiments of the present disclosure may include but not limited to mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 4 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 4, an electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may be randomly accessed according to a program stored in a read-only memory (ROM) 602 or loaded from a storage device 608. Various appropriate actions and processes are executed by programs in the memory (RAM) 603 . In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored. The processing device 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604 .

Typically, the following devices can be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 607 such as a computer; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 4 shows electronic device 600 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server can communicate using any currently known or future network protocols such as HTTP (HyperText Transfer Protocol, Hypertext Transfer Protocol), and can communicate with digital data in any form or medium The communication (eg, communication network) interconnections. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires a training sample set, wherein each training sample in the training sample set contains A training image and a polyp labeling category corresponding to the training image; according to the training image and the image generation model, a generated image and a restored image corresponding to the training image are obtained, wherein the image generation model includes a first generator and a second generator Two generators, the first generator is used to generate the generated image according to the training image, and the second generator is used to generate the restored image according to the generated image; according to the training image and the generated image , determine the first distribution distance corresponding to the training image and the generated image, wherein the first distribution distance is used to represent the difference between the distribution of the training image and the distribution of the generated image; according to the The first distribution distance, the training image, the generated image, the restored image, and the polyp label category corresponding to the training image determine the target loss of the image generation model, wherein the target loss includes according to the The first distribution loss determined by the first distribution distance, the first distribution loss is negatively correlated with the first distribution distance; when the update condition is met, the image generation model is generated according to the target loss The parameters are updated.

Alternatively, the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: receives the polyp image to be identified; inputs the polyp image into the polyp identification model , to obtain the recognition result of the polyp image, wherein the training sample set corresponding to the polyp recognition model includes the original sample and the generated sample generated according to the original sample and the first generator in the image generation model, the image The generation model is obtained by training based on the training method of the polyp image generation model described in the first aspect, the original sample includes the original image and the polyp label category corresponding to the original image, and the generation sample includes the polyp generated based on the original image Generate an image and annotate the polyp category corresponding to the original image.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages - such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, using an Internet service provider to connected via the Internet).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the module does not constitute a limitation of the module itself under certain circumstances, for example, the obtaining module may also be described as "a module for obtaining the training sample set".

The functions described herein above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, Example 1 provides a method for training a polyp image generation model, the method comprising:

According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein, according to the first distribution distance, the training image, the generated image, the restored image, and the training The polyp annotation category corresponding to the image determines the target loss of the image generation model, including:

determining a generation loss of the image generation model according to the training image and a restored image corresponding to the training image;

Determine the prediction loss of the image generation model based on the polyp labeling category corresponding to the training image and the polyp prediction category corresponding to the generated image generated based on the training image;

According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the training sample set contains training samples corresponding to multiple polyp labeling categories;

The determining the target loss of the image generation model according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image further includes:

According to the generated images under various polyp labeling categories, for the generated images under any two polyp labeling categories, determine the second distribution distance corresponding to the generated images under the two polyp labeling categories, wherein the second distribution distance is used to represent the difference between distributions of generated images belonging to different polyp annotation categories;

The determining the target loss according to the generation loss, the prediction loss and the distribution loss includes:

determining a second distribution difference of the image generation model according to the second distribution distance;

A weighted sum of the generation loss, the prediction loss, the first distribution loss, and the second distribution loss is determined as the target loss.

According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1, wherein, according to the training image and the generated image, determining the first distribution corresponding to the training image and the generated image distance, including:

For training images and generated images under the same polyp labeling category, determine the transmission distance between the training images, the transmission distance between the generated images, and the transmission distance between the training images and the generated images;

The first distribution distance is determined according to the transmission distance between the training image and the generated image, the transmission distance between the training images, and the transmission distance between the generated images.

According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4, wherein the transmission distance is determined by the following formula:

According to one or more embodiments of the present disclosure, Example 6 provides a polyp identification method, wherein the method includes:

receiving the polyp image to be identified;

Inputting the polyp image into a polyp recognition model to obtain a recognition result of the polyp image, wherein the training sample set corresponding to the polyp recognition model includes an original sample, and according to the original sample and the first generated image in the image generation model The generation sample generated by the machine, the image generation model is obtained by training based on the training method of the polyp image generation model described in any one of examples 1-5, and the original sample includes the original image and the corresponding image of the original image. The polyp labeling category, the generated samples include a generated image generated based on the original image and a polyp labeling category corresponding to the original image.

According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 6, wherein the polyp recognition model is trained in the following manner:

According to one or more embodiments of the present disclosure, Example 8 provides a training device for a polyp image generation model, the device comprising:

An update module, configured to update the parameters of the image generation model according to the target loss when an update condition is satisfied.

According to one or more embodiments of the present disclosure, Example 9 provides a polyp identification device, the device comprising:

A receiving module, configured to receive an image of a polyp to be identified;

An identification module, configured to input the polyp image into a polyp identification model to obtain an identification result of the polyp image, wherein the training sample set corresponding to the polyp identification model includes original samples, and a model is generated according to the original samples and images The generation sample generated by the first generator in the above, the image generation model is obtained by training based on the training method of the polyp image generation model described in any one of examples 1-5, and the original sample includes the original image and the The polyp labeling category corresponding to the original image, the generated sample includes a generated image generated based on the original image and the polyp labeling category corresponding to the original image.

According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing device, the steps of any one of the methods described in Examples 1-7 are implemented .

According to one or more embodiments of the present disclosure, Example 11 provides an electronic device, including:

a storage device on which a computer program is stored;

A processing device configured to execute the computer program in the storage device to implement the steps of any one of the methods in Examples 1-7.

The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principles. Those skilled in the art should understand that the disclosure scope involved in this disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but also covers the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of equivalent features. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

In addition, while operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or performed in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while the above discussion contains several specific implementation details, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

Claims

A kind of training method of polyp image generation model, it is characterized in that, described method comprises:

Obtaining a training sample set, wherein each training sample in the training sample set includes a training image and a polyp label category corresponding to the training image;

According to the training image and the image generation model, a generated image and a restored image corresponding to the training image are obtained, wherein the image generation model includes a first generator and a second generator, and the first generator is used to The training image is used to generate the generated image, and the second generator is used to generate the restored image according to the generated image;

According to the training image and the generated image, determine a first distribution distance corresponding to the training image and the generated image, wherein the first distribution distance is used to represent the distribution of the training image and the generated image The difference between the distributions of ;

According to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, determine the target loss of the image generation model, wherein the target loss includes The first distribution loss determined according to the first distribution distance, the first distribution loss and the first distribution distance are negatively correlated;

If the updating condition is met, the parameters of the image generation model are updated according to the target loss.
The method according to claim 1, wherein, according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, the determined Target losses for the image generation models described above, including:

determining a generation loss of the image generation model according to the training image and a restored image corresponding to the training image;

Determine the prediction loss of the image generation model based on the polyp labeling category corresponding to the training image and the polyp prediction category corresponding to the generated image generated based on the training image;

determining a negative value of the first distribution distance as the first distribution loss;

The target loss is determined based on the generation loss, the prediction loss, and the first distribution loss.
The method according to claim 2, wherein the training sample set includes training samples corresponding to multiple polyp labeling categories;

The determining the target loss of the image generation model according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image further includes:

According to the generated images under various polyp labeling categories, for the generated images under any two polyp labeling categories, determine the second distribution distance corresponding to the generated images under the two polyp labeling categories, wherein the second distribution distance is used to represent the difference between distributions of generated images belonging to different polyp annotation categories;

The determining the target loss according to the generation loss, the prediction loss and the distribution loss includes:

determining a second distribution difference of the image generation model according to the second distribution distance;

A weighted sum of the generation loss, the prediction loss, the first distribution loss, and the second distribution loss is determined as the target loss.
The method according to claim 1, wherein the determining the first distribution distance corresponding to the training image and the generated image according to the training image and the generated image comprises:

For training images and generated images under the same polyp labeling category, determine the transmission distance between the training images, the transmission distance between the generated images, and the transmission distance between the training images and the generated images;

The first distribution distance is determined according to the transmission distance between the training image and the generated image, the transmission distance between the training images, and the transmission distance between the generated images.
The method according to claim 4, wherein the transmission distance is determined by the following formula:

Wherein, W c (X 1 , X 2 ) is used to represent the transmission distance between image X 1 and image X 2 ;

φ(X 1 ) is used to represent the feature image extracted from the image X 1 ;

φ(X 2 ) is used to represent the feature image extracted from the image X 2 ;

P 1 is used to represent the distribution corresponding to the image X 1 ; P 2 is used to represent the distribution corresponding to the image X 2 ;

Π(P 1 ,P 2 ) is used to represent the entire joint distribution formed by distribution P 1 and distribution P 2 ;

c(X 1 , X 2 ) is used to represent the transmission cost between the image X 1 and the image X 2 .
A polyp identification method, characterized in that the method comprises:

receiving the polyp image to be identified;

Inputting the polyp image into a polyp recognition model to obtain a recognition result of the polyp image, wherein the training sample set corresponding to the polyp recognition model includes an original sample, and according to the original sample and the first generated image in the image generation model The generation sample generated by the machine, the image generation model is obtained by training based on the training method of the polyp image generation model described in any one of claims 1-5, and the original sample includes the original image corresponding to the original image polyp labeling category, the generated samples include a generated image generated based on the original image and a polyp labeling category corresponding to the original image.
The method according to claim 6, wherein the polyp recognition model is trained in the following manner:

Preprocessing the target training image in the training sample set to obtain a processed image, wherein the preprocessing includes nonlinear transformation and/or local pixel shuffling, and the target training image includes the original image and the generated image;

Pre-training the polyp recognition model by using the processed image as a model input and using the target training image as a target output to obtain a pre-trained polyp recognition model;

The target training image is used as a model input, and the polyp label category corresponding to the target training image is used as a target output to train the pre-trained polyp recognition model to obtain a trained polyp recognition model.
A kind of training device of polyp image generation model, it is characterized in that, described device comprises:

An acquisition module, configured to acquire a training sample set, wherein each training sample in the training sample set includes a training image and a polyp label category corresponding to the training image;

A generation module, configured to obtain a generated image and a restored image corresponding to the training image according to the training image and the image generation model, wherein the image generation model includes a first generator and a second generator, and the first The generator is used to generate the generated image according to the training image, and the second generator is used to generate the restored image according to the generated image;

A first determining module, configured to determine a first distribution distance corresponding to the training image and the generated image according to the training image and the generated image, wherein the first distribution distance is used to represent the training image The difference between the distribution of and the distribution of the generated images;

The second determination module is configured to determine the target loss of the image generation model according to the first distribution distance, the training image, the generated image, the restored image, and the polyp labeling category corresponding to the training image, Wherein, the target loss includes a first distribution loss determined according to the first distribution distance, and the first distribution loss is negatively correlated with the first distribution distance;

An update module, configured to update the parameters of the image generation model according to the target loss when an update condition is satisfied.
A polyp identification device, characterized in that the device comprises:

A receiving module, configured to receive an image of a polyp to be identified;

An identification module, configured to input the polyp image into a polyp identification model to obtain an identification result of the polyp image, wherein the training sample set corresponding to the polyp identification model includes original samples, and a model is generated according to the original samples and images The generated sample generated by the first generator in the method, the image generation model is obtained by training based on the training method of the polyp image generation model described in any one of claims 1-5, and the original sample includes the original image and The polyp labeling category corresponding to the original image, and the generated sample includes a generated image generated based on the original image and the polyp labeling category corresponding to the original image.
A computer-readable medium, on which a computer program is stored, characterized in that, when the program is executed by a processing device, the steps of the method described in any one of claims 1-7 are implemented.
An electronic device, characterized in that it comprises:

a storage device on which a computer program is stored;

A processing device configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1-7.