WO2022105308A1 - 一种基于生成对抗级联网络增广图像的方法 - Google Patents

一种基于生成对抗级联网络增广图像的方法 Download PDF

Info

Publication number
WO2022105308A1
WO2022105308A1 PCT/CN2021/110525 CN2021110525W WO2022105308A1 WO 2022105308 A1 WO2022105308 A1 WO 2022105308A1 CN 2021110525 W CN2021110525 W CN 2021110525W WO 2022105308 A1 WO2022105308 A1 WO 2022105308A1
Authority
WO
WIPO (PCT)
Prior art keywords
level
image
real
discriminator
generator
Prior art date
Application number
PCT/CN2021/110525
Other languages
English (en)
French (fr)
Inventor
袁杰
程裕家
金志斌
周雪
Original Assignee
南京大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京大学 filed Critical 南京大学
Publication of WO2022105308A1 publication Critical patent/WO2022105308A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4023—Scaling of whole images or parts thereof, e.g. expanding or contracting based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the present invention relates to the field of ultrasound image analysis, in particular to a method for augmenting images based on a generative adversarial cascade network.
  • image research of deep learning large-scale datasets are usually relied on to avoid the overfitting problem.
  • traditional image augmentation methods are usually used for image augmentation, such as multiple cropping, adding Gaussian noise, and grayscale equalization.
  • the amount of image data is often insufficient, or the types of images are not rich enough.
  • Using a good image augmentation method can often play a multiplier or even decisive role; but at the same time, a single image augmentation method It may also lead to overfitting of the network, resulting in poor generalization performance of the network; in addition, the images generated by the single-stage generative adversarial network have problems such as high similarity between images and low resolution.
  • the technical problem to be solved by the present invention is to provide a method for augmenting images based on a generative adversarial cascade network based on the deficiencies of the prior art.
  • the present invention discloses a method for augmenting images based on a generative confrontation cascade network, comprising the following steps:
  • Step 1 delineate a region of interest from the original image I ori and crop it to obtain a cropped image I cut ;
  • Step 2 preprocessing the cropped image I cut to augment the image to obtain the augmented data set S cut ;
  • Step 3 using the augmented data set S cut to train the I-level generative adversarial network, and verify the test, and save the trained I-level discriminator and I-level generator;
  • Step 4 Load the trained first-level generator, infer an image by inputting random noise, use the upsampling method to post-process the inferred image, make an image with a size of W*L, and add it to a new image.
  • dataset S I In dataset S I ;
  • Step 5 Use the new data set S I produced in step 4 and the cropped image I cut as the training set of the level II generative adversarial network together, carry out the level II generative adversarial network training, and verify the test, save the trained Class II discriminator and class II generator;
  • Step 6 load the trained level II generator, input the data set S I processed in step 4, and infer the augmented image I des , the size of the augmented image I des is W*L;
  • the image data set S I with certain prior information is used as the input of the second-level generator, and the obtained augmented image I des is more diverse than the images obtained by the traditional augmentation method.
  • the step 1 includes: selecting an image sub-block including the target area from the original image I ori and cutting it to obtain a cropped image I cut , the cropped image The size of I cut is W*L, and the image sub-block including the target area is the region of interest of the original image.
  • the step 2 includes: the preprocessing refers to performing multi-sampling on the cropped image I cut to augment the image to obtain an augmented data set S cut ; Multi-sampling plays the role of image augmentation, increases the number of images in the dataset, and reduces the training difficulty of the subsequent level I generative adversarial network.
  • the step 3 includes:
  • Step 3-1 in the level I generative adversarial network, the level I generator is connected in series with the level I discriminator, input random noise, and after the level I generator, generate the level I generated image;
  • Step 3-2 train the I-level discriminator, add the data set S cut obtained through the step 2 to the real image data set S I,real , and input the real image data set S I,real to the I-level generation.
  • the training of the I-level discriminator consists of two parts, the first part is the real image data set S I, the images in real are judged to be true, and the second part is that the generated images of the I-level are judged to be false, in these two
  • the loss function value output by the I-level discriminator is returned to the I-level discriminator, and the network parameters of the I-level generator do not participate in the update, only the network parameters of the I-level discriminator are updated;
  • Step 3-3 train the level I generator, input the level I generated image into the level I discriminator, and set the label of the level I generated image to true; when the level I generator is trained, the level I discriminator is fixed, Return the loss function value output by the I-level generator to the I-level generator, and only update the network parameters of the I-level generator and keep the I-level discriminator network parameters unchanged;
  • step 3-4 the trained level I discriminator and the level I generator are generated from the network parameters of the trained level I generator and the network parameters of the level I discriminator.
  • the step 4 includes:
  • Step 4-1 input the random noise into the I-level generator described in step 3, and perform inference to obtain I-level generated images;
  • Step 4-2 utilize the method of upsampling to restore the I-level generated image obtained in step 4-1 to the image size W*L after the cropping in step 1; the upsampling is upsampling based on interpolation;
  • step 4-3 the interpolated image is processed by means of normalization, histogram equalization and contrast increase, and the processed image is added to the new data set S I.
  • the step 5 includes:
  • Step 5-1 input the new dataset S I produced in step 4 into the level II generator of the level II generative adversarial network, and after passing through the level II generator, generate the level II generated image;
  • Step 5-2 train the level II discriminator, add the cropped image I cut in step 1 to the real image dataset S II,real , and input the real image dataset S II,real into the level II generative adversarial network , and the second-level generated image is used as the input image of the second-level discriminator; the label of the image in the real image data set S II, real is set to true, and the label of the second-level generated image is set to false;
  • the training of the level discriminator consists of two parts, the first part is the real image dataset S II, the images in the real are judged to be true, and the second part is that the generated images of the second level are judged to be false, in these two processes , the loss function value output by the level II discriminator is returned to the level II discriminator, the network parameters of the level II generator do not participate in the update, and only the network parameters of the level II discriminator are updated;
  • Step 5-3 train the level II generator, input the level II generated image into the level II discriminator, and set the label of the level II generated image to true; when the level II generator is trained, the level II discriminator is fixed, Return the loss function value output by the level II generator to the level II generator, and only update the network parameters of the level II generator and keep the network parameters of the level II discriminator unchanged;
  • step 5-4 the trained level II discriminator and the level II generator are generated from the network parameters of the trained level II generator and the network parameters of the level II discriminator.
  • the loss function values output by the level I discriminator in the steps 3-2 and 3-3 both include the loss function value of the level I discriminator and the loss function value of the level I generator.
  • the loss function value of the I-level discriminator includes the sum of the error calculation result of the image in the real image data set S I, real and the error calculation result of the I-level generated image, and the calculation formula is as follows:
  • loss fake criterion(fake out ,fake label )
  • loss real is the loss function value obtained by the I-level discriminator for the images in the real image dataset S I
  • real loss fake is the loss function value obtained by the I-level discriminator for the I-level generated images
  • real label is the real Image data set S I, the label of the image in real , the label is 1 at this time
  • real out is the specific image in the real image data set SI , real
  • fake out is the specific image of the generated image at level I
  • the fake label is level I
  • the label of the generated image, the label is 0 at this time
  • loss d is the overall loss function of the I-level discriminator obtained after the results of the generated image and the real image dataset S I, real in the real image data set S I, real, criterion represents the loss function calculation method
  • the loss function value of the I-level generator is calculated by combining the labels of the images in the real image dataset S I, real with the I-level generated images, and the calculation formula is as follows:
  • loss g is the loss function of the level I generator
  • output represents the generated image of level I
  • fake_label represents the label of the image in the real image dataset S I, real , and the label is 0 at this time.
  • both the level I generator and the level I discriminator select the Adam optimizer to update the network parameters.
  • the loss function values output by the level II discriminator in the steps 5-2 and 5-3 both include the loss function value of the level II discriminator and the loss function value of the level II generator.
  • the loss function value of the Class II discriminator includes the sum of the error calculation results of the images in the real image data set S II, real and the error calculation results of the generated images of Class II, and the calculation formula is as follows:
  • loss II,raal criterion(real II,out ,real II,label )
  • loss II real is the loss function value obtained by the level II discriminator for the images in the real image dataset S II
  • real , loss II, fake is the loss function value obtained by the level II discriminator for the generated image at level II
  • real II, label is the label of the image in the real image data set S II, real , the label is 1 at this time
  • real II, out is the specific image in the real image data set S II, real
  • fake II, out is generated by level II
  • the specific image of the image, fake II, label is the label of the second-level generated image, the label is 0 at this time
  • loss II, d is generated through the second-level image and the real image data set S II,
  • the result of the image in the real image data set S II, real is summed up.
  • the obtained overall loss function of the Class II discriminator, criterion represents the calculation method of the loss function
  • the loss function of the level II generator is obtained by combining the labels of the images in the real image dataset S II, real with the generated images of level II, and the calculation formula is as follows:
  • loss II, g is the loss function of the level II generator
  • output II represents the generated image of level II
  • fake_label II represents the label of the image in the real image dataset S II, real , and the label is 0 at this time.
  • both the level II generator and the level II discriminator select the Adam optimizer to update the network parameters.
  • the augmented image is generated by cascading the two-level generative adversarial network, and the image generated by the I-level generative adversarial network is post-processed and input to the II-level generative adversarial network, that is, the second-level generative adversarial network is used to generate an image.
  • the present invention solves the problem of insufficient training data for deep learning research using existing image samples, and avoids the problem of network overfitting caused by the traditional augmentation method; at the same time, it solves the problem of single-stage generation confrontation
  • the problems of high similarity and low resolution between the images generated by the network improve the generalization performance of the network.
  • FIG. 1 is a schematic diagram of the workflow of a level I generative adversarial network in a method for augmenting an image based on a generative adversarial cascade network provided in an embodiment of the present invention
  • FIG. 2 is a schematic diagram of the workflow of a level II generative adversarial network in a method for augmenting an image based on a generative adversarial cascade network provided in part by an embodiment of the present invention
  • FIG. 3 is a schematic diagram of the neural network architecture of a level I generator (G) and a level I discriminator (D) of a level I generative adversarial network in a method for augmenting an image based on a generative adversarial cascade network provided in an embodiment of the present invention. ;
  • FIG. 4 is a schematic diagram of the neural network architecture of a level II generator (G) and a level II discriminator (D) of a level II generative adversarial network in a method for augmenting images based on a generative adversarial cascade network provided in an embodiment of the present invention ;
  • the embodiment of the present invention discloses a method for augmenting images based on a generated adversarial cascade network.
  • the method is applied to the augmentation research of ultrasonic images of arthritis. This has led to a delay in related research on ultrasound images.
  • Step 1 delineate the region of interest from the original image I ori and crop, and obtain the cropped image I cut ; in this embodiment, Matlab software can be used to delineate the region of interest of the original image and carry out definite size cropping, thereby obtaining the cropped image I cut .
  • Step 2 Perform preprocessing on the cropped image I cut to augment the image to obtain an augmented data set S cut ; in this embodiment, the preprocessing refers to image sampling.
  • Step 3 Use the augmented data set S cut to train the level I generative adversarial network, verify and test, and save the trained level I discriminator and level I generator.
  • the Generative Adversarial Networks is a network formed by a generator (G) and a discriminator (D);
  • the generative adversarial cascade network is a combination of two generative adversarial networks (I A combined network formed by cascading the first-level generative adversarial network and the second-level generative adversarial network).
  • Step 4 Load the trained first-level generator, infer an image by inputting random noise, use the upsampling method to post-process the inferred image, make an image with a size of W*L, and add it to a new image.
  • dataset S I contains 720 images.
  • Step 5 Take the new data set S I produced in step 4 and the cropped image I cut in step 1 as the training set of the second-level generative adversarial network together, carry out the level-II generative adversarial network training, and verify the test, save the trained Class II discriminator and class II generator.
  • the training set of the level II generative adversarial network refers to the data set jointly obtained by the cropped image I cut in step 1 and the new data set S I produced in step 4.
  • Step 6 Load the trained level II generator, input the data set S I processed in step 4, and infer the augmented image I des , the size of the augmented image I des is W*L.
  • the level II generator refers to the generator saved in step 5.
  • the step 1 includes: selecting an image sub-block containing a target area from the original image I ori and cropping, and obtaining a cropped image sub-block.
  • the size of the cropped image I cut is W*L
  • the image sub-block including the target region is the region of interest of the original image.
  • the used original image I ori is an image of the diseased part of arthritis acquired by a medical ultrasound imaging device, there are 720 images in total, and the imaging depth of the images is different according to the different acquisition devices.
  • the resolution of the original image is 1024*768, and the unit is pixel.
  • Matlab software is used to classify the target area contained in the original image as synovial membrane.
  • the image sub-block at the location is cropped.
  • the size of the cropped image I cut is 512*128, and the cropped image I cut is used as a training sample, with a total of 720 images.
  • the step 2 includes: performing multi-sampling on the cropped image I cut to augment the image, and obtain the augmented data set S cut ;
  • the cropped 512*128 image is sampled into an image with a size of 64*64 according to the method of 8 sampling in the width direction and 2 sampling in the height direction.
  • the number of image samples is increased by 16 times of the original by means of multi-sampling.
  • the step 3 includes:
  • Step 3-1 in the level I generative adversarial network, the level I generator is connected in series with the level I discriminator, input random noise, and after the level I generator, generate the level I generated image;
  • Step 3-2 train the I-level discriminator, add the data set S cut obtained by the step 2 to the real image data set S I, real , and input the real image data set S I, real to the I-level generation.
  • the adversarial network together with the I-level generated image as the input image of the I-level discriminator; the real image dataset S I, the label of the image in real is set to true, and the label of the I-level generated image is set to be false;
  • the training of the I-level discriminator consists of two parts, the first part is the real image data set S I, the images in real are judged to be true, and the second part is that the I-level generated images are judged to be false, in these two
  • the loss function value output by the I-level discriminator is sent back to the I-level discriminator, and the network parameters of the I-level generator do not participate in the update, only the network parameters of the I-level discriminator are updated;
  • Step 3-3 train the I-level generator, input the I-level generated image into the I-level discriminator, and set the label of the I-level generated image to be true; when the I-level generator is trained, the I-level discriminator is fixed, Return the loss function value output by the I-level generator to the I-level generator, and only update the network parameters of the I-level generator and keep the I-level discriminator network parameters unchanged;
  • the loss function values output by the I-level discriminator in steps 3-2 and 3-3 both include the loss function value of the I-level discriminator and the loss function value of the I-level generator; the loss of the I-level discriminator
  • the function consists of two parts, which is the sum of the error calculation result for the real image and the error calculation result for the generated image.
  • the calculation method of the loss function is BCEloss:
  • loss fake criterion(fake out , fake label )
  • loss real is the loss function value obtained by the I-level discriminator for the images in the real image dataset S I
  • real loss fake is the loss function value obtained by the I-level discriminator for the I-level generated image
  • real label is real Image dataset S I, the label of the image in real , the label is 1 at this time
  • real out is the real image dataset S I, the specific image of the image in real
  • fake out is the specific image of the generated image at level I
  • the fake label is The label of the I-level generated image, which is 0 at this time
  • loss d is the overall loss function of the I-level discriminator obtained after the results of the I-level generated image and the real image dataset S I
  • real images are summarized, criterion, criterion
  • the calculation method representing the loss function is essentially a functor, and the calculation method used in this embodiment is BCEloss.
  • the loss function of the I-level generator is based on the real image dataset S I, the label of the image in the real is combined with the I-level generated image, and the loss function is calculated by BCEloss.
  • the real image label is in the I-level. It is recorded as 0 in the generative adversarial network:
  • loss g is the loss function of the level I generator
  • output represents the generated image of level I
  • fake_label represents the label of the image in the real image dataset S I, real
  • the label is 0 at this time
  • criterion represents the calculation method of the loss function
  • both the level I generator and the level I discriminator need to select appropriate optimization algorithms to ensure that the loss function of the level I generator and the loss function of the level I discriminator converge to the maximum value at the same time. , to prevent the divergence of the loss function value.
  • Adam optimizer is selected for the I-level generator and I-level discriminator to update the network parameters.
  • step 3-4 the trained level I discriminator and the level I generator are generated from the network parameters of the trained level I generator and the network parameters of the level I discriminator.
  • the augmented data set S cut in the step 2 is used as a training sample, and the training is performed through a level I generative adversarial network.
  • the basic flow chart of the level I generative adversarial network is shown in Figure 1
  • the neural network architecture of the level I generator (G) and the level I discriminator (D) is shown in Figure 3.
  • a set of trained I-level discriminators and I-level generators are obtained by training all samples, and the network parameters of I-level discriminators are shown in Table 1.
  • the network parameters of the first-level generator are shown in Table 2.
  • Convolutional layer Conv2d-1 [32, 64, 32, 32]
  • Convolutional layer Conv2d-2 [32, 128, 16, 16]
  • Convolutional layer Conv2d-3 [32, 256, 8, 8]
  • Convolutional layer Conv2d-4 [32, 512, 4, 4] Linear-5 [32, 1]
  • Deconvolution layer ConvTranspose2d-1 [32, 512, 4, 4] Deconvolution layer ConvTranspose2d-2 [32, 256, 8, 8] Deconvolution layer ConvTranspose2d-3 [32, 128, 16, 16] Deconvolution layer ConvTranspose2d-4 [32, 64, 32, 32] Deconvolution layer ConvTranspose2d-5 [32, 3, 64, 64]
  • the step 4 includes:
  • Step 4-1 input the random noise into the I-level generator after training described in step 3, and perform inference to obtain I-level generated images;
  • Step 4-2 utilize the method of upsampling to restore the I-level generated image obtained in step 4-1 to the image size W*L after the cropping in step 1; the upsampling is upsampling based on interpolation;
  • the size of the image generated by the I-level generator is 64*64
  • the interp2 function of Matlab is used to perform 8-fold and 2-fold interpolation in the length and height directions of the image, respectively, to restore the image to a size of 512*128.
  • step 4-3 the interpolated image is processed by means of normalization, histogram equalization and contrast increase, and the processed image is added to the new data set S I.
  • an image with a size of 512*128 after interpolation is normalized and histogram equalized by using Matlab, so as to increase the contrast of the image and improve the image quality.
  • the step 5 includes:
  • Step 5-1 input the new dataset S I produced in step 4 into the level II generator of the level II generative adversarial network, and after passing through the level II generator, generate the level II generated image;
  • Step 5-2 train the level II discriminator, add the cropped image I cut in step 1 to the real image dataset S II,real , and input the real image dataset S II,real into the level II generative adversarial network , and the second-level generated image is used as the input image of the second-level discriminator; the label of the image in the real image data set S II, real is set to true, and the label of the second-level generated image is set to false;
  • the training of the level discriminator consists of two parts, the first part is the real image dataset S II, the images in the real are judged to be true, and the second part is that the generated images of the second level are judged to be false, in these two processes , the loss function value output by the level II discriminator is returned to the level II discriminator, the network parameters of the level II generator do not participate in the update, and only the network parameters of the level II discriminator are updated;
  • Step 5-3 train the II-level generator, input the II-level generated image into the II-level discriminator, and set the label of the II-level generated image to true; when the II-level generator is trained, the II-level discriminator is fixed, Return the loss function value output by the level II generator to the level II generator, and only update the network parameters of the level II generator and keep the network parameters of the level II discriminator unchanged;
  • the loss function values output by the level II discriminator in steps 5-2 and 5-3 both include the loss function value of the level II discriminator and the loss function value of the level II generator; the level II discriminator
  • the loss function consists of two parts, the sum of the error calculation result for the real image and the error calculation result for the generated image. Among them, under Pytorch, the calculation method of the loss function is BCEloss:
  • loss II, d loss II , real + loss II, fake
  • loss lI real are the loss function values obtained by the II-level discriminator for the images in the real image dataset S II
  • real , loss II, fake are the loss function values obtained by the II-level discriminator for the II-level generated images
  • real II, label is the real image data set S II, the label of the image in real, the label is 1 at this time
  • real II, out is the specific image of the image in the real image data set S II, real ; fake II, out is II
  • the specific image of the level-generated image, fake II, label is the label of the level-II generated image, the label is 0 at this time
  • loss II, d is the result of the generated image through the level-II and the real image dataset S II, real .
  • the overall loss function of the level II discriminator obtained later, criterion represents the calculation method of the loss function, which is essentially a functor, and the calculation method used in this embodiment is BCEloss.
  • the loss function of the level II generator is based on the real image dataset S II, the labels of the images in real and the generated images of level II are combined, and the loss function is calculated by BCEloss.
  • the labels of the real images are at level II It is recorded as 0 in the generative adversarial network:
  • loss II, g is the loss function of the level II generator
  • output II represents the generated image of level II
  • fake_label II represents the label of the image in the real image dataset S II, real
  • the label is 0 at this time
  • criterion represents the loss function
  • the calculation method of is essentially a functor, and the calculation method used in this embodiment is BCEloss.
  • both the level II generator and the level II discriminator need to select an appropriate optimization algorithm to ensure that the loss function of the level II generator and the loss function of the level II discriminator converge to the maximum value at the same time. , to prevent the divergence of the loss function value.
  • the second-level generator and the second-level discriminator use the Adam optimizer to update the network parameters.
  • step 5-4 the trained level II discriminator and the level II generator are generated from the network parameters of the trained level II generator and the network parameters of the level II discriminator.
  • the 512*128 image generated and processed in step 4 is used as the input of the level II generator of the level II generative adversarial network, and it is used together with the 512*128 image cropped in step 1 as level II generation
  • the input of the level II discriminator of the adversarial network is used for adversarial training of the level II generative adversarial network.
  • the basic flow chart of the level II generative adversarial network is shown in Figure 2.
  • the neural network architecture is shown in Figure 4. Using the neural network architecture of the level II generator and level II discriminator, a set of trained level II discriminators and level II generators are obtained by training all samples.
  • the network parameters of the level II discriminator are shown in Table 3.
  • the network parameters of the Level II generator are shown in Table 4.
  • Convolutional layer Conv2d-1 [16, 32, 64, 256] Pooling layer AvgPool2d-2 [16, 32, 32, 128] Convolutional layer Conv2d-3 [16, 64, 32, 128] Pooling layer AvgPool2d-4 [16, 64, 16, 64] Convolutional layer Conv2d-5 [16, 128, 16, 64] Pooling layer AvgPool2d-6 [16, 128, 8, 32] Convolutional layer Conv2d-7 [16, 256, 8, 32] Pooling layer AvgPool2d-8 [16, 256, 4, 16] Linear-9 [16, 1]
  • Convolutional layer Conv2d-1 [16, 200, 128, 512]
  • Convolutional layer Conv2d-2 [16, 100, 128, 512]
  • Convolutional layer Conv2d-3 [16, 50, 128, 512]
  • Convolutional layer Conv2d-4 [16, 25, 128, 512]
  • Convolutional layer Conv2d-5 [16, 3, 128, 512]
  • the new data set S I produced in the step 4 is input into the level II generator trained in the step 5, so that Carry out inference to obtain 512*128 high-resolution images to achieve the purpose of data augmentation.
  • the images generated by the I-level generative adversarial network are post-processed and input to the II-level generative adversarial network to generate new images with obvious differences and high resolution, which improves the difference and resolution of image augmentation.
  • the present invention solves the problem of insufficient training data for deep learning research using existing image samples, and avoids the problem of network overfitting caused by being limited to traditional augmentation methods;
  • the problems of high similarity and low resolution between the images generated by the level I generative adversarial network improve the generalization performance of the network.
  • the present invention proposes a method for augmenting an image based on a generated adversarial cascade network. It should be pointed out that the type of ultrasound equipment required does not limit the patent; the scale, size and resolution of the collected ultrasound images do not limit the patent; The captured image content does not limit this patent. It should be pointed out that for those skilled in the art, some improvements and modifications can be made without departing from the principles of the invention, and these should also be regarded as the protection scope of the present invention. In addition, each component that is not specified in this embodiment can be implemented by the prior art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于生成对抗级联网络增广图像的方法。包括：从原始图像I ori圈定感兴趣区域并裁剪，获得裁剪后的图像I cut；对I cut通过预处理获得增广后的数据集S cut；利用数据集S cut进行Ⅰ级生成对抗网络的训练；加载训练好的Ⅰ级生成器，输入随机噪声推理出图像，并对生成的图像通过上采样处理后，制作成新的数据集S Ⅰ；利用数据集S Ⅰ与I cut作为Ⅱ级生成对抗网络的训练数据集，进行Ⅱ级生成对抗网络的训练；加载训练好的Ⅱ级生成器，将数据集S Ⅰ输入Ⅱ级生成器，推理出所需的增广图像I des。本发明在对图像增广时，解决了Ⅰ级生成对抗网络中生成图像差异性小和分辨率低的问题，在增广图像的同时提高了网络的泛化性能。

Description

一种基于生成对抗级联网络增广图像的方法 技术领域
本发明涉及超声图像分析领域，尤其涉及一种基于生成对抗级联网络增广图像的方法。
背景技术
在深度学习的图像研究中，通常都依赖于大规模的数据集以避免过拟合问题的发生。当图像数据量严重不足时，通常采用传统图像增广方式进行图像增广，例如多次裁剪、添加高斯噪声、灰度均衡等。
这些传统图像增广方法在对现有数据集实现扩充的同时也给网络的训练带来了过拟合的风险。原因是通过这些传统的图像增广方法得到的图像与原始图像相关性极高，而且单级生成对抗网络生成的图像也存在一定的相似性且分辨率较低，这些方法并不能明显提高数据集样本的多样性。随着增广数据量的增加，数据集中雷同的数据项越来越多，最终导致网络过拟合，泛化性能差。
在深度学习领域中常常存在着图像数据量不足，或者图像种类不够丰富等情况，使用良好的图像增广方法往往能起到事半功倍甚至是决定性的作用；但与此同时，单一的图像增广方式也有可能会导致网络的过拟合，致使网络的泛化性能差；除此之外，单级生成对抗网络生成的图像存在图像之间相似度较高且分辨率低等问题。
发明内容
发明目的：本发明所要解决的技术问题是针对现有技术的不足，提供一种基于生成对抗级联网络增广图像的方法。
为了解决上述技术问题，本发明公开了一种基于生成对抗级联网络增广图像的方法，包括如下步骤：
步骤1，从原始图像I ori圈定感兴趣区域并裁剪，获得裁剪后的图像I cut；
步骤2，对裁剪的图像I cut进行预处理以增广图像，获得增广后的数据集S cut；
步骤3，利用所述增广后的数据集S cut进行Ⅰ级生成对抗网络的训练，并验证测试，保存训练好的Ⅰ级判别器和Ⅰ级生成器；
步骤4，加载所述训练好的Ⅰ级生成器，通过输入随机噪声推理出图像，对推理出 的图像运用上采样的方法进行后处理，制作成尺寸为W*L的图像并添加至新的数据集S Ⅰ中；
步骤5，将步骤4制作的新的数据集S Ⅰ与所述裁剪后的图像I cut共同作为Ⅱ级生成对抗网络的训练集，进行Ⅱ级生成对抗网络训练，并验证测试，保存训练好的Ⅱ级判别器和Ⅱ级生成器；
步骤6，加载所述训练好的Ⅱ级生成器，输入经步骤4处理后的数据集S Ⅰ，推理出增广图像I des，所述增广图像I des的尺寸为W*L；将具有一定先验信息的图像数据集S Ⅰ作为Ⅱ级生成器的输入，获得的增广图像I des相对于传统增广方式获得的图像更具有多样性。
进一步地，在一种实现方式中，所述步骤1包括：从所述原始图像I ori中选择包含目标区域的图像子块并进行裁剪，获得裁剪后的图像I cut，所述裁剪后的图像I cut的尺寸为W*L，所述包含目标区域的图像子块即原始图像的感兴趣区域。
进一步地，在一种实现方式中，所述步骤2包括：所述预处理指对所述裁剪后的图像I cut进行多抽样以增广图像，获得增广后的数据集S cut；对图像进行多抽样起到图像增广的作用，增加了数据集图像数量，降低后续Ⅰ级生成对抗网络的训练难度。
进一步地，在一种实现方式中，所述步骤3包括：
步骤3-1，所述Ⅰ级生成对抗网络中所述Ⅰ级生成器后串接Ⅰ级判别器，输入随机噪声，经由Ⅰ级生成器后，生成Ⅰ级生成图像；
步骤3-2，训练Ⅰ级判别器，将通过所述步骤2获得的数据集S cut添加到真实图像数据集S Ⅰ,real，将所述真实图像数据集S Ⅰ,real输入到Ⅰ级生成对抗网络中，和所述Ⅰ级生成图像一起作为Ⅰ级判别器的输入图像；将所述真实图像数据集S Ⅰ,real中图像的标签设置为真，所述Ⅰ级生成图像的标签设置为假；Ⅰ级判别器的训练由两部分组成，第一部分是所述真实图像数据集S Ⅰ,real中的图像判别为真，第二部分是所述Ⅰ级生成图像判别为假，在这两个过程中，将Ⅰ级判别器输出的损失函数值回传至Ⅰ级判别器，Ⅰ级生成器的网络参数不参与更新，只更新所述Ⅰ级判别器的网络参数；
步骤3-3，训练Ⅰ级生成器，将Ⅰ级生成图像输入到Ⅰ级判别器中，将所述Ⅰ级生成图像的标签设置为真；Ⅰ级生成器训练时，Ⅰ级判别器固定，将Ⅰ级生成器输出的损失函数值回传至Ⅰ级生成器，只更新所述Ⅰ级生成器的网络参数而保持Ⅰ级判别器的网络参数不 变；
步骤3-4，由训练好的Ⅰ级生成器的网络参数和Ⅰ级判别器的网络参数生成训练好的Ⅰ级判别器和Ⅰ级生成器。
进一步地，在一种实现方式中，所述步骤4包括：
步骤4-1，将随机噪声输入步骤3所述Ⅰ级生成器，进行推理获得I级生成图像；
步骤4-2，利用上采样的方法将步骤4-1中获得的I级生成图像还原成步骤1裁剪后的图像尺寸W*L；所述上采样为基于插值的上采样；
步骤4-3，对插值后的图像用归一化、直方图均衡的方法和增加对比度进行处理，将处理后的图像添加至新的数据集S Ⅰ中。
进一步地，在一种实现方式中，所述步骤5包括：
步骤5-1，将步骤4制作的新的数据集S Ⅰ，输入Ⅱ级生成对抗网络的Ⅱ级生成器，经由Ⅱ级生成器后，生成Ⅱ级生成图像；
步骤5-2，训练Ⅱ级判别器，将步骤1裁剪后的图像I cut添加到真实图像数据集S Ⅱ,real，将所述真实图像数据集S Ⅱ,real输入到Ⅱ级生成对抗网络中，和所述Ⅱ级生成图像一起作为Ⅱ级判别器的输入图像；将所述真实图像数据集S Ⅱ,real中图像的标签设置为真，所述Ⅱ级生成图像的标签设置为假；Ⅱ级判别器的训练由两部分组成，第一部分是所述真实图像数据集S Ⅱ,real中的图像判别为真，第二部分是所述Ⅱ级生成图像判别为假，在这两个过程中，将Ⅱ级判别器输出的损失函数值回传至Ⅱ级判别器，Ⅱ级生成器的网络参数不参与更新，只更新所述Ⅱ级判别器的网络参数；
步骤5-3，训练Ⅱ级生成器，将Ⅱ级生成图像输入到Ⅱ级判别器中，将所述Ⅱ级生成图像的标签设置为真；Ⅱ级生成器训练时，Ⅱ级判别器固定，将Ⅱ级生成器输出的损失函数值回传至Ⅱ级生成器，只更新所述Ⅱ级生成器的网络参数而保持Ⅱ级判别器的网络参数不变；
步骤5-4，由训练好的Ⅱ级生成器的网络参数和Ⅱ级判别器的网络参数生成训练好的Ⅱ级判别器和Ⅱ级生成器。
进一步地，在一种实现方式中，所述步骤3-2和步骤3-3中Ⅰ级判别器输出的损失函数值均包括Ⅰ级判别器的损失函数值和Ⅰ级生成器的损失函数值；所述Ⅰ级判别器的损失函数值包括对所述真实图像数据集S Ⅰ,real中图像的误差计算结果和对Ⅰ级生成图像的 误差计算结果之和，计算公式如下：
loss real＝criterion(real out,real label)
loss fake＝criterion(fake out,fake label)
loss d＝loss real+loss fake
其中，loss real为Ⅰ级判别器对真实图像数据集S Ⅰ,real中图像得出的损失函数值，loss fake为Ⅰ级判别器对Ⅰ级生成图像得出的损失函数值，real label为真实图像数据集S Ⅰ,real中图像的标签，该标签此时为1，real out为真实图像数据集S Ⅰ,real中具体图像；fake out为Ⅰ级生成图像的具体图像，fake label为Ⅰ级生成图像的标签，该标签此时为0，loss d是经由Ⅰ级生成图像和真实图像数据集S Ⅰ,real中图像的结果汇总之后所得到的Ⅰ级判别器的整体损失函数，criterion代表损失函数的计算方法；
所述Ⅰ级生成器的损失函数值是由真实图像数据集S Ⅰ,real中图像的标签和Ⅰ级生成图像相结合计算获得，计算公式如下：
loss g＝criterion(output,fack_label)
其中，loss g是Ⅰ级生成器的损失函数，output代表Ⅰ级生成图像，fack_label代表真实图像数据集S Ⅰ,real中图像的标签，该标签此时为0。
进一步地，在一种实现方式中，所述步骤3中，Ⅰ级生成器和Ⅰ级判别器均选用Adam优化器进行网络参数更新。
进一步地，在一种实现方式中，所述步骤5-2和步骤5-3中Ⅱ级判别器输出的损失函数值均包含Ⅱ级判别器的损失函数值和Ⅱ级生成器的损失函数值；所述Ⅱ级判别器的损失函数值包括对真实图像数据集S Ⅱ,real中图像的误差计算结果和对Ⅱ级生成图像的误差计算结果之和，计算公式如下：
loss Ⅱ,raal＝criterion(real Ⅱ,out,real Ⅱ,label)
loss Ⅱ,fake＝criterion(fake Ⅱ,out,fake Ⅱ,label)
loss Ⅱ,d＝loss Ⅱ,real+loss Ⅱ,fake
其中，loss Ⅱ,real为Ⅱ级判别器对真实图像数据集S Ⅱ,real中图像得出的损失函数值，loss Ⅱ,fake为Ⅱ级判别器对Ⅱ级生成图像得出的损失函数值，real Ⅱ,label为真实图像数据集S Ⅱ,real中图像的标签，该标签此时为1，real Ⅱ,out为真实图像数据集S Ⅱ,real中具体图像；fake Ⅱ,out为Ⅱ级生成图像的具体图像，fake Ⅱ,label为Ⅱ级生成图像的标签，该标签此时为0，loss Ⅱ,d是经由Ⅱ级生成图像和真实图像数据集S Ⅱ,real中图像的结果汇总之后所得到的Ⅱ级判别器的整体损失函数，criterion代表损失函数的计算方法；
所述Ⅱ级生成器的损失函数是由真实图像数据集S Ⅱ,real中图像的标签和Ⅱ级生成图像相结合计算获得，计算公式如下：
loss Ⅱ,g＝criterion(output Ⅱ,fack_label Ⅱ)
其中，loss Ⅱ,g是Ⅱ级生成器的损失函数，output Ⅱ代表Ⅱ级生成图像，fack_label Ⅱ代表真实图像数据集S Ⅱ,real中图像的标签，该标签此时为0。
进一步地，在一种实现方式中，所述步骤5中，Ⅱ级生成器和Ⅱ级判别器均选用Adam优化器进行网络参数更新。
有益效果：本发明中，利用两级生成对抗网络级联生成增广图像，将Ⅰ级生成对抗网络生成的图像经过后处理，输入到Ⅱ级生成对抗网络，即利用Ⅱ级生成对抗网络将生成对抗网络中常以随机噪声作为输入改为将具有一定先验信息的图像作为输入，生成具有明显差异性和高分辨率的新图像，提高了图像增广的差异性和分辨率；相对于现有技术，本发明解决了仅利用现有图像样本进行深度学习研究的训练数据量不足的问题，并且避免了局限于传统增广方式而造成的网络过拟合问题；同时，解决了单级生成对抗网络生成的图像之间相似度较高且分辨率低等问题，提高了网络的泛化性能。
附图说明
为了更清楚地说明本发明的技术方案，下面将对实施例中所需要使用的附图作简 单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例部分提供的一种基于生成对抗级联网络增广图像的方法中Ⅰ级生成对抗网络的工作流程示意图；
图2是本发明实施例部分提供的一种基于生成对抗级联网络增广图像的方法中Ⅱ级生成对抗网络的工作流程示意图；
图3是本发明实施例部分提供的一种基于生成对抗级联网络增广图像的方法中Ⅰ级生成对抗网络的Ⅰ级生成器(G)和Ⅰ级判别器(D)的神经网络架构示意图；
图4是本发明实施例部分提供的一种基于生成对抗级联网络增广图像的方法中Ⅱ级生成对抗网络的Ⅱ级生成器(G)和Ⅱ级判别器(D)的神经网络架构示意图；
具体实施方式
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。
本发明实施例公开一种基于生成对抗级联网络增广图像的方法，本方法应用于关节炎超声图像的增广研究，由于该病的患病人群较少，可供研究的样本不足，进而导致超声图像的相关研究迟缓。
本实施例所述的一种基于生成对抗级联网络增广图像的方法，包括如下步骤：
步骤1，从原始图像I ori圈定感兴趣区域并裁剪，获得裁剪后的图像I cut；本实施例中，可以采用Matlab软件圈定原始图像感兴趣区域并进行确定尺寸裁剪，从而获得裁剪后的图像I cut。
步骤2，对裁剪的图像I cut进行预处理增广图像，获得增广后的数据集S cut；本实施例中，所述预处理指图像抽样。
步骤3，利用所述增广后的数据集S cut进行Ⅰ级生成对抗网络的训练，并验证测试，保存训练好的Ⅰ级判别器和Ⅰ级生成器。本实施例中，所述生成对抗网络(Generative Adversarial Networks,GAN)为通过生成器(G)和判别器(D)形成的网络；所述生成对抗级联网络是将两个生成对抗网络(Ⅰ级生成对抗网络和Ⅱ级生成对抗网络)经过级联形成的组合网络。
步骤4，加载所述训练好的Ⅰ级生成器，通过输入随机噪声推理出图像，对推理出 的图像运用上采样的方法进行后处理，制作成尺寸为W*L的图像并添加至新的数据集S Ⅰ中。本实施例中，数据集S Ⅰ中包含720幅图像。
步骤5，将步骤4制作的新的数据集S Ⅰ与步骤1裁剪后的图像I cut共同作为Ⅱ级生成对抗网络的训练集，进行Ⅱ级生成对抗网络训练，并验证测试，保存训练好的Ⅱ级判别器和Ⅱ级生成器。本步骤中，所述的Ⅱ级生成对抗网络的训练集指步骤1中裁剪后的图像I cut和步骤4中制作的新的数据集S Ⅰ共同获得的数据集。
步骤6，加载所述训练好的Ⅱ级生成器，输入经步骤4处理后的数据集S Ⅰ，推理出增广图像I des，所述增广图像I des的尺寸为W*L。本步骤中，所述的Ⅱ级生成器指步骤5中保存的生成器。
本实施例所述的一种基于生成对抗级联网络增广图像的方法中，所述步骤1包括：从所述原始图像I ori中选择包含目标区域的图像子块并进行裁剪，获得裁剪后的图像I cut，所述裁剪后的图像I cut的尺寸为W*L，所述包含目标区域的图像子块即原始图像的感兴趣区域。
具体的，本步骤中，后续的处理都针对这个感兴趣区域以减少处理时间、提高精度。本实施例中，使用的原始图像I ori是由医学超声成像设备采集得到的关节炎患病部位图像，共有720幅，图像的成像深度根据采集设备的不同而有所区别。所述原始图像的分辨率为1024*768，单位为像素，为了剔除所述原始图像的无效区域，减少生成对抗网络的计算量和计算时间，利用Matlab软件对原始图像中包含目标区域为滑膜所在位置的图像子块进行裁剪，裁剪后的图像I cut的尺寸为512*128，将裁剪后的图像I cut作为训练样本，共有720幅。
本实施例中，所述步骤2包括：对裁剪的图像I cut进行多抽样以增广图像，获得增广后的数据集S cut；
具体的，本实施例中，对裁剪得到的512*128的图像分别按照宽度方向8抽样和高度方向2抽样的方法抽样成大小为64*64的图像。本实施例中，通过多抽样的方式，使图像样本数量增广为原来的16倍。
本实施例中，所述步骤3包括：
步骤3-1，所述Ⅰ级生成对抗网络中所述Ⅰ级生成器后串接Ⅰ级判别器，输入随机噪声，经由Ⅰ级生成器后，生成Ⅰ级生成图像；
步骤3-2，训练I级判别器，将通过所述步骤2获得的数据集S cut添加到真实图像数据集S I，real，将所述真实图像数据集S I，real输入到I级生成对抗网络中，和所述I级生成图像一起作为I级判别器的输入图像；将所述真实图像数据集S I，real中图像的标签设置为真，所述I级生成图像的标签设置为假；I级判别器的训练由两部分组成，第一部分是所述真实图像数据集S I，real中的图像判别为真，第二部分是所述I级生成图像判别为假，在这两个过程中，将I级判别器输出的损失函数值回传至I级判别器，I级生成器的网络参数不参与更新，只更新所述I级判别器的网络参数；
步骤3-3，训练I级生成器，将I级生成图像输入到I级判别器中，将所述I级生成图像的标签设置为真；I级生成器训练时，I级判别器固定，将I级生成器输出的损失函数值回传至I级生成器，只更新所述I级生成器的网络参数而保持I级判别器的网络参数不变；
本实施例中，步骤3-2和步骤3-3中I级判别器输出的损失函数值均包含I级判别器的损失函数值和I级生成器的损失函数值；I级判别器的损失函数包括两个部分，为对真实图像的误差计算结果和对生成图像的误差计算结果之和。其中，在Pytorch下，损失函数的计算方法为BCEloss：
loss real＝criterion(real out，real label)
loss fake＝criterion(fake out，fake label)
loss d＝loss real+loss fake
其中，loss real为I级判别器对真实图像数据集S I，real中图像得出的损失函数值，loss fake为I级判别器对I级生成图像得出的损失函数值，real label为真实图像数据集S I，real中图像的标签，该标签此时为1，real out为真实图像数据集S I，real中图像的具体图像；fake out为I级生成图像的具体图像，fake label为I级生成图像的标签，该标签此时为0，loss d是经由I级生成图像和真实图像数据集S I，real中图像的结果汇总之后所得到的I级判别器的整体损失函数，criterion代表损失函数的计算方法，本质上是一种仿函数，本实施例中使用的计算方法是BCEloss。
I级生成器的损失函数则是以真实图像数据集S I，real中图像的标签和I级生成图像相结合，以BCEloss来计算损失函数，本实施例中，真实图像的标签即在I级生成对抗网 络中记为0：
loss g＝criterion(output,fack_label)
其中，loss g是Ⅰ级生成器的损失函数，output代表Ⅰ级生成图像，fack_label代表真实图像数据集S Ⅰ,real中图像的标签，该标签此时为0，criterion代表损失函数的计算方法，本质上是一种仿函数，本实施例中使用的计算方法是BCEloss。
此外，由于卷积神经网络的需要，Ⅰ级生成器和Ⅰ级判别器均需要选择合适的优化算法，保证Ⅰ级生成器的损失函数和Ⅰ级判别器的损失函数在极大值收敛的同时，防止损失函数值的发散。具体的实现上，Ⅰ级生成器和Ⅰ级判别器选用了Adam优化器进行网络参数更新。本实施例中，每轮训练送入Ⅰ级生成对抗网络中的训练样本的数量batch_size＝32，训练迭代次数epoch＝200，学习速率lr＝0.0002，输入Ⅰ级生成器的随机噪声的维度z_dimension＝100。
步骤3-4，由训练好的Ⅰ级生成器的网络参数和Ⅰ级判别器的网络参数生成训练好的Ⅰ级判别器和Ⅰ级生成器。
本实施例中，所述步骤3中利用步骤2中增广后的数据集S cut作为训练样本，通过Ⅰ级生成对抗网络进行训练。其中，Ⅰ级生成对抗网络的基本流程图如图1所示，Ⅰ级生成器(G)和Ⅰ级判别器(D)的神经网络架构如图3所示。运用所述Ⅰ级生成器和I级判别器的神经网络架构，通过训练所有样本得到一组训练后的Ⅰ级判别器和Ⅰ级生成器，其中Ⅰ级判别器的网络参数如表1所示，Ⅰ级生成器的网络参数如表2所示。
表1 Ⅰ级判别器的网络参数
网络层类型 网络输出尺寸
卷积层Conv2d-1 [32，64，32，32]
卷积层Conv2d-2 [32，128，16，16]
卷积层Conv2d-3 [32，256，8，8]
卷积层Conv2d-4 [32，512，4，4]
Linear-5 [32，1]
表2 Ⅰ级生成器的网络参数
网络层类型 网络输出尺寸
逆卷积层ConvTranspose2d-1 [32，512，4，4]
逆卷积层ConvTranspose2d-2 [32，256，8，8]
逆卷积层ConvTranspose2d-3 [32，128，16，16]
逆卷积层ConvTranspose2d-4 [32，64，32，32]
逆卷积层ConvTranspose2d-5 [32，3，64，64]
本实施例所述的一种基于生成对抗级联网络增广图像的方法中，所述步骤4包括：
步骤4-1，将随机噪声输入步骤3所述训练后的Ⅰ级生成器，进行推理获得I级生成图像；
步骤4-2，利用上采样的方法将步骤4-1中获得的I级生成图像还原成步骤1裁剪后的图像尺寸W*L；所述上采样为基于插值的上采样；
本实施例中，Ⅰ级生成器生成图像尺寸为64*64，利用Matlab的interp2函数在图像长度方向和高度方向分别做8倍以及2倍插值，将图像还原为512*128的尺寸。
步骤4-3，对插值后的图像用归一化、直方图均衡的方法和增加对比度进行处理，将处理后的图像添加至新的数据集S Ⅰ中。
本实施例中，将插值后的大小为512*128的图像利用Matlab做归一化、直方图均衡处理，增加图像的对比度，提升图像质量。
本实施例所述的一种基于生成对抗级联网络增广图像的方法中，所述步骤5包括：
步骤5-1，将步骤4制作的新的数据集S Ⅰ，输入Ⅱ级生成对抗网络的Ⅱ级生成器，经由Ⅱ级生成器后，生成Ⅱ级生成图像；
步骤5-2，训练Ⅱ级判别器，将步骤1裁剪后的图像I cut添加到真实图像数据集S Ⅱ,real，将所述真实图像数据集S Ⅱ,real输入到Ⅱ级生成对抗网络中，和所述Ⅱ级生成图像一起作为Ⅱ级判别器的输入图像；将所述真实图像数据集S Ⅱ,real中图像的标签设置为真，所述Ⅱ级生成图像的标签设置为假；Ⅱ级判别器的训练由两部分组成，第一部分是所述真实图像数据集S Ⅱ,real中的图像判别为真，第二部分是所述Ⅱ级生成图像判别为假，在这两个过程中，将Ⅱ级判别器输出的损失函数值回传至Ⅱ级判别器，Ⅱ级生成器的网络参数不参与更新，只更新所述Ⅱ级判别器的网络参数；
步骤5-3，训练II级生成器，将II级生成图像输入到II级判别器中，将所述II级生成图像的标签设置为真；II级生成器训练时，II级判别器固定，将II级生成器输出的损失函数值回传至II级生成器，只更新所述II级生成器的网络参数而保持II级判别器的网络参数不变；
本实施例中，步骤5-2和步骤5-3中II级判别器输出的损失函数值均包含II级判别器的损失函数值和II级生成器的损失函数值；所述II级判别器的损失函数包括两个部分，为对真实图像的误差计算结果和对生成图像的误差计算结果之和。其中，在Pytorch下，损失函数的计算方法为BCEloss：
loss II，real＝criterion(real II，out，real II，label)
loss II，fake＝criterion(fake II，out，fake II，label)
loss II，d＝loss II， real+loss II，fake
其中，loss lI，real为II级判别器对真实图像数据集S II，real中图像得出的损失函数值，loss II，fake为II级判别器对II级生成图像得出的损失函数值，real II，label为真实图像数据集S II，real中图像的标签，该标签此时为1，real II，out为真实图像数据集S II，real中图像的具体图像；fake II，out为II级生成图像的具体图像，fake II，label为II级生成图像的标签，该标签此时为0，loss II，d是经由II级生成图像和真实图像数据集S II，real中图像的结果汇总之后所得到的II级判别器的整体损失函数，criterion代表损失函数的计算方法，本质上是一种仿函数，本实施例中使用的计算方法是BCEloss。
II级生成器的损失函数则是以真实图像数据集S II，real中图像的标签和II级生成图像相结合，以BCEloss来计算损失函数，本实施例中，真实图像的标签即在II级生成对抗网络中记为0：
loss II，g＝criterion(output II，fack_label II)
其中，loss II，g是II级生成器的损失函数，output II代表II级生成图像，fack_label II 代表真实图像数据集S Ⅱ,real中图像的标签，该标签此时为0，criterion代表损失函数的计算方法，本质上是一种仿函数，本实施例中使用的计算方法是BCEloss。
此外，由于卷积神经网络的需要，Ⅱ级生成器和Ⅱ级判别器均需要选择合适的优化算法，保证Ⅱ级生成器的损失函数和Ⅱ级判别器的损失函数在极大值收敛的同时，防止损失函数值的发散。具体的实现上，Ⅱ级生成器和Ⅱ级判别器选用了Adam优化器进行网络参数更新。本实施例中，每轮训练送入Ⅱ级生成对抗网络中的训练样本的数量batch_size＝16，训练迭代次数epoch＝200，学习速率lr＝0.0003。
步骤5-4，由训练好的Ⅱ级生成器的网络参数和Ⅱ级判别器的网络参数生成训练好的Ⅱ级判别器和Ⅱ级生成器。
本实施例中，将步骤4生成并处理后的512*128的图像作为Ⅱ级生成对抗网络的Ⅱ级生成器的输入，将其与步骤1裁剪后的512*128的图像共同作为Ⅱ级生成对抗网络的Ⅱ级判别器的输入，进行Ⅱ级生成对抗网络的对抗训练，其中，Ⅱ级生成对抗网络的基本流程图如图2所示，Ⅱ级生成器(G)和Ⅱ级判别器(D)的神经网络架构如图4所示。运用所述Ⅱ级生成器和Ⅱ级判别器的神经网络架构，通过训练所有样本得到一组训练后的Ⅱ级判别器和Ⅱ级生成器，其中Ⅱ级判别器的网络参数如表3所示，Ⅱ级生成器的网络参数如表4所示。
表3 Ⅱ级判别器网络参数
网络层类型 网络输出尺寸
卷积层Conv2d-1 [16，32，64，256]
池化层AvgPool2d-2 [16，32，32，128]
卷积层Conv2d-3 [16，64，32，128]
池化层AvgPool2d-4 [16，64，16，64]
卷积层Conv2d-5 [16，128，16，64]
池化层AvgPool2d-6 [16，128，8，32]
卷积层Conv2d-7 [16，256，8，32]
池化层AvgPool2d-8 [16，256，4，16]
Linear-9 [16，1]
表4 Ⅱ级生成器网络参数
网络层类型 网络输出尺寸
卷积层Conv2d-1 [16，200，128，512]
卷积层Conv2d-2 [16，100，128，512]
卷积层Conv2d-3 [16，50，128，512]
卷积层Conv2d-4 [16，25，128，512]
卷积层Conv2d-5 [16，3，128，512]
本实施例所述的一种基于生成对抗级联网络增广图像的方法中，所述步骤6中将步骤4制作的新的数据集S Ⅰ，输入步骤5训练后的Ⅱ级生成器，从而进行推理，得到512*128的高分辨率图像，达到数据增广的目的。
本发明中，将Ⅰ级生成对抗网络生成的图像经过后处理，输入到Ⅱ级生成对抗网络，生成具有明显差异性和高分辨率的新图像，提高了图像增广的差异性和分辨率，相对于现有技术，本发明解决了仅利用现有图像样本进行深度学习研究的训练数据量不足的问题，并且避免了局限于传统增广方式而造成的网络过拟合问题；同时，解决了Ⅰ级生成对抗网络生成的图像之间相似度较高且分辨率低等问题，提高了网络的泛化性能。
本发明提出了一种基于生成对抗级联网络增广图像的方法，应当指出，所需的超声设备种类不对本专利构成限制；所采集的超声图像规模、尺寸和分辨率不对本专利构成限制；所采集的图像内容不对本专利构成限制。应当指出，对于本技术领域的普通人员来说，在不脱离发明原理的前提下还可以做出若干改进和润饰，这些也应视为本发明的保护范围。另外，本实施例中未明确的各组成部分均可用现有技术加以实现。

Claims (10)

  1. 一种基于生成对抗级联网络增广图像的方法，其特征在于，包括如下步骤：
    步骤1，从原始图像I ori圈定感兴趣区域并裁剪，获得裁剪后的图像I cut，所述裁剪后的图像I cut的尺寸为W*L；
    步骤2，对裁剪的图像I cut进行预处理以增广图像，获得增广后的数据集S cut；
    步骤3，利用所述增广后的数据集S cut进行Ⅰ级生成对抗网络的训练，并验证测试，保存训练好的Ⅰ级判别器和Ⅰ级生成器；
    步骤4，加载所述训练好的Ⅰ级生成器，通过输入随机噪声推理出图像，对推理出的图像运用上采样的方法进行后处理，制作成尺寸为W*L的图像并添加至新的数据集S Ⅰ中；
    步骤5，将新的数据集S Ⅰ与所述裁剪后的图像I cut共同作为Ⅱ级生成对抗网络的训练集，进行Ⅱ级生成对抗网络训练，并验证测试，保存训练好的Ⅱ级判别器和Ⅱ级生成器；
    步骤6，加载所述训练好的Ⅱ级生成器，输入经步骤4处理后的数据集S Ⅰ，推理出增广图像I des，所述增广图像I des的尺寸为W*L。
  2. 根据权利要求1所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤1包括：从所述原始图像I ori中选择包含目标区域的图像子块并进行裁剪，获得裁剪后的图像I cut，所述裁剪后的图像I cut的尺寸为W*L，所述包含目标区域的图像子块即原始图像的感兴趣区域。
  3. 根据权利要求1所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤2中，所述预处理指对所述裁剪后的图像I cut进行多抽样以增广图像，获得增广后的数据集S cut。
  4. 根据权利要求1所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤3包括：
    步骤3-1，所述Ⅰ级生成对抗网络中所述Ⅰ级生成器后串接Ⅰ级判别器，输入随机噪声，经由Ⅰ级生成器后，生成Ⅰ级生成图像；
    步骤3-2，训练Ⅰ级判别器，将通过所述步骤2获得的数据集S cut添加到真实图像数据集S Ⅰ,real，将所述真实图像数据集S Ⅰ,real输入到Ⅰ级生成对抗网络中，和所述Ⅰ级生成图像一起作为Ⅰ级判别器的输入图像；将所述真实图像数据集S Ⅰ,real中图像的标签设置为真， 所述Ⅰ级生成图像的标签设置为假；Ⅰ级判别器的训练由两部分组成，第一部分是所述真实图像数据集S Ⅰ,real中的图像判别为真，第二部分是所述Ⅰ级生成图像判别为假，在这两个过程中，将Ⅰ级判别器输出的损失函数值回传至Ⅰ级判别器，Ⅰ级生成器的网络参数不参与更新，只更新所述Ⅰ级判别器的网络参数；
    步骤3-3，训练Ⅰ级生成器，将Ⅰ级生成图像输入到Ⅰ级判别器中，将所述Ⅰ级生成图像的标签设置为真；Ⅰ级生成器训练时，Ⅰ级判别器固定，将Ⅰ级生成器输出的损失函数值回传至Ⅰ级生成器，只更新所述Ⅰ级生成器的网络参数而保持Ⅰ级判别器的网络参数不变；
    步骤3-4，由训练好的Ⅰ级生成器的网络参数和Ⅰ级判别器的网络参数生成训练好的Ⅰ级判别器和Ⅰ级生成器。
  5. 根据权利要求1所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤4包括：
    步骤4-1，将随机噪声输入步骤3所述训练好的Ⅰ级生成器，进行推理获得I级生成图像；
    步骤4-2，利用上采样的方法将步骤4-1中获得的I级生成图像还原成步骤1裁剪后的图像尺寸W*L；所述上采样为基于插值的上采样；
    步骤4-3，对插值后的图像用归一化、直方图均衡的方法和增加对比度进行处理，将处理后的图像添加至新的数据集S Ⅰ中。
  6. 根据权利要求1所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤5包括：
    步骤5-1，将步骤4制作的新的数据集S Ⅰ，输入Ⅱ级生成对抗网络的Ⅱ级生成器，经由Ⅱ级生成器后，生成Ⅱ级生成图像；
    步骤5-2，训练Ⅱ级判别器，将步骤1裁剪后的图像I cut添加到真实图像数据集S Ⅱ,real，将所述真实图像数据集S Ⅱ,real输入到Ⅱ级生成对抗网络中，和所述Ⅱ级生成图像一起作为Ⅱ级判别器的输入图像；将所述真实图像数据集S Ⅱ,real中图像的标签设置为真，所述Ⅱ级生成图像的标签设置为假；Ⅱ级判别器的训练由两部分组成，第一部分是所述真实图像数据集S Ⅱ,real中的图像判别为真，第二部分是所述Ⅱ级生成图像判别为假，在这两个过程中，将Ⅱ级判别器输出的损失函数值回传至Ⅱ级判别器，Ⅱ级生成器的网络参数不 参与更新，只更新所述Ⅱ级判别器的网络参数；
    步骤5-3，训练Ⅱ级生成器，将Ⅱ级生成图像输入到Ⅱ级判别器中，将所述Ⅱ级生成图像的标签设置为真；Ⅱ级生成器训练时，Ⅱ级判别器固定，将Ⅱ级生成器输出的损失函数值回传至Ⅱ级生成器，只更新所述Ⅱ级生成器的网络参数而保持Ⅱ级判别器的网络参数不变；
    步骤5-4，由训练好的Ⅱ级生成器的网络参数和Ⅱ级判别器的网络参数生成训练好的Ⅱ级判别器和Ⅱ级生成器。
  7. 根据权利要求4所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤3-2和步骤3-3中Ⅰ级判别器输出的损失函数值均包括Ⅰ级判别器的损失函数值和Ⅰ级生成器的损失函数值；所述Ⅰ级判别器的损失函数值包括对所述真实图像数据集S Ⅰ,real中图像的误差计算结果和对Ⅰ级生成图像的误差计算结果之和，计算公式如下：
    loss real＝criterion(real out,real label)
    loss fake＝criterion(fake out,fake label)
    loss d＝loss real+loss fake
    其中，loss real为Ⅰ级判别器对真实图像数据集S Ⅰ,real中图像得出的损失函数值，loss fake为Ⅰ级判别器对Ⅰ级生成图像得出的损失函数值，real label为真实图像数据集S Ⅰ,real中图像的标签，该标签此时为1，real out为真实图像数据集S Ⅰ,real中具体图像；fake out为Ⅰ级生成图像的具体图像，fake label为Ⅰ级生成图像的标签，该标签此时为0，loss d是经由Ⅰ级生成图像和真实图像数据集S Ⅰ,real中图像的结果汇总之后所得到的Ⅰ级判别器的整体损失函数，criterion代表损失函数的计算方法；
    所述Ⅰ级生成器的损失函数值是由真实图像数据集S Ⅰ,real中图像的标签和Ⅰ级生成图像相结合计算获得，计算公式如下：
    loss g＝criterion(output,fack_label)
    其中，loss g是Ⅰ级生成器的损失函数，output代表Ⅰ级生成图像，fack_label代表真实图像数据集S Ⅰ,real中图像的标签，该标签此时为0。
  8. 根据权利要求7所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤3中，Ⅰ级生成器和Ⅰ级判别器均选用Adam优化器进行网络参数更新。
  9. 根据权利要求6所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤5-2和步骤5-3中Ⅱ级判别器输出的损失函数值均包含Ⅱ级判别器的损失函数值和Ⅱ级生成器的损失函数值；所述Ⅱ级判别器的损失函数值包括对真实图像数据集S Ⅱ,real中图像的误差计算结果和对Ⅱ级生成图像的误差计算结果之和，计算公式如下：
    loss Ⅱ,real＝criterion(real Ⅱ,out,real Ⅱ,label)
    loss Ⅱ,fake＝criterion(fake Ⅱ,out,fake Ⅱ,label)
    loss Ⅱ,d＝loss Ⅱ,real+loss Ⅱ,fake
    其中，loss Ⅱ,real为Ⅱ级判别器对真实图像数据集S Ⅱ,real中图像得出的损失函数值，loss Ⅱ,fake为Ⅱ级判别器对Ⅱ级生成图像得出的损失函数值，real Ⅱ,label为真实图像数据集S Ⅱ,real中图像的标签，该标签此时为1，real Ⅱ,out为真实图像数据集S Ⅱ,real中具体图像；fake Ⅱ,out为Ⅱ级生成图像的具体图像，fake Ⅱ,label为Ⅱ级生成图像的标签，该标签此时为0，loss Ⅱ,d是经由Ⅱ级生成图像和真实图像数据集S Ⅱ,real中图像的结果汇总之后所得到的Ⅱ级判别器的整体损失函数，criterion代表损失函数的计算方法；
    所述Ⅱ级生成器的损失函数是由真实图像数据集S Ⅱ,real中图像的标签和Ⅱ级生成图像相结合计算获得，计算公式如下：
    loss Ⅱ,g＝criterion(output Ⅱ,fack_label Ⅱ)
    其中，loss Ⅱ,g是Ⅱ级生成器的损失函数，output Ⅱ代表Ⅱ级生成图像，fack_label Ⅱ代表真实图像数据集S Ⅱ,real中图像的标签，该标签此时为0。
  10. 根据权利要求9所述的一种基于生成对抗级联网络增广图像的方法，其特征在于，所述步骤5中，Ⅱ级生成器和Ⅱ级判别器均选用Adam优化器进行网络参数更新。
PCT/CN2021/110525 2020-11-20 2021-08-04 一种基于生成对抗级联网络增广图像的方法 WO2022105308A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011307427.8 2020-11-20
CN202011307427.8A CN112396110B (zh) 2020-11-20 2020-11-20 一种基于生成对抗级联网络增广图像的方法

Publications (1)

Publication Number Publication Date
WO2022105308A1 true WO2022105308A1 (zh) 2022-05-27

Family

ID=74606700

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/110525 WO2022105308A1 (zh) 2020-11-20 2021-08-04 一种基于生成对抗级联网络增广图像的方法

Country Status (2)

Country Link
CN (1) CN112396110B (zh)
WO (1) WO2022105308A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114863225A (zh) * 2022-07-06 2022-08-05 腾讯科技（深圳）有限公司 图像处理模型训练方法、生成方法、装置、设备及介质
CN115661134A (zh) * 2022-12-08 2023-01-31 浙江霖研精密科技有限公司 基于生成对抗网络的缺陷样本生成系统、方法和存储介质
CN115661001A (zh) * 2022-12-14 2023-01-31 临沂大学 基于生成对抗网络的单通道煤岩图像增强方法
CN115797163A (zh) * 2023-02-13 2023-03-14 中国人民解放军火箭军工程大学 一种基于遥感影像的目标数据跨域反演增广方法
CN116012258A (zh) * 2023-02-14 2023-04-25 山东大学 一种基于循环生成对抗网络的图像和谐化方法
CN116310348A (zh) * 2023-05-18 2023-06-23 安徽农业大学 一种基于cgan的语义分割数据集增广的方法
CN117409008A (zh) * 2023-12-15 2024-01-16 华东交通大学 一种设备图像生成方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112396110B (zh) * 2020-11-20 2024-02-02 南京大学 一种基于生成对抗级联网络增广图像的方法
CN113702439A (zh) * 2021-08-20 2021-11-26 浙江科技学院 一种基于迭代生成稀疏主成分模型的红外无损检测方法
CN114494081B (zh) * 2022-04-01 2022-07-05 武汉大学 一种无人机遥感测绘图像增强方法
CN117094986B (zh) * 2023-10-13 2024-04-05 中山大学深圳研究院 基于小样本的自适应缺陷检测方法及终端设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190066281A1 (en) * 2017-08-24 2019-02-28 Siemens Healthcare Gmbh Synthesizing and Segmenting Cross-Domain Medical Images
CN111008940A (zh) * 2019-11-27 2020-04-14 北京科技大学 一种图像增强方法及装置
CN111325236A (zh) * 2020-01-21 2020-06-23 南京大学 一种基于卷积神经网络的超声图像分类方法
CN111563841A (zh) * 2019-11-13 2020-08-21 南京信息工程大学 一种基于生成对抗网络的高分辨率图像生成方法
CN112396110A (zh) * 2020-11-20 2021-02-23 南京大学 一种基于生成对抗级联网络增广图像的方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10825219B2 (en) * 2018-03-22 2020-11-03 Northeastern University Segmentation guided image generation with adversarial networks
US11501438B2 (en) * 2018-04-26 2022-11-15 Elekta, Inc. Cone-beam CT image enhancement using generative adversarial networks
CN109948660A (zh) * 2019-02-26 2019-06-28 长沙理工大学 一种改进辅助分类器gan的图像分类方法
CN110516561B (zh) * 2019-08-05 2022-12-06 西安电子科技大学 基于dcgan和cnn的sar图像目标识别方法
CN111028146B (zh) * 2019-11-06 2022-03-18 武汉理工大学 基于双判别器的生成对抗网络的图像超分辨率方法
CN111476717B (zh) * 2020-04-07 2023-03-24 西安电子科技大学 基于自注意力生成对抗网络的人脸图像超分辨重建方法
CN111539467A (zh) * 2020-04-17 2020-08-14 北京工业大学 基于生成对抗网络为医疗影像数据集做数据增广的gan网络架构及方法
CN111861906B (zh) * 2020-06-22 2023-10-31 长安大学 一种路面裂缝图像虚拟增广模型建立及图像虚拟增广方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190066281A1 (en) * 2017-08-24 2019-02-28 Siemens Healthcare Gmbh Synthesizing and Segmenting Cross-Domain Medical Images
CN111563841A (zh) * 2019-11-13 2020-08-21 南京信息工程大学 一种基于生成对抗网络的高分辨率图像生成方法
CN111008940A (zh) * 2019-11-27 2020-04-14 北京科技大学 一种图像增强方法及装置
CN111325236A (zh) * 2020-01-21 2020-06-23 南京大学 一种基于卷积神经网络的超声图像分类方法
CN112396110A (zh) * 2020-11-20 2021-02-23 南京大学 一种基于生成对抗级联网络增广图像的方法

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114863225B (zh) * 2022-07-06 2022-10-04 腾讯科技（深圳）有限公司 图像处理模型训练方法、生成方法、装置、设备及介质
CN114863225A (zh) * 2022-07-06 2022-08-05 腾讯科技（深圳）有限公司 图像处理模型训练方法、生成方法、装置、设备及介质
CN115661134B (zh) * 2022-12-08 2023-03-21 浙江霖研精密科技有限公司 基于生成对抗网络的缺陷样本生成系统、方法和存储介质
CN115661134A (zh) * 2022-12-08 2023-01-31 浙江霖研精密科技有限公司 基于生成对抗网络的缺陷样本生成系统、方法和存储介质
CN115661001B (zh) * 2022-12-14 2023-04-07 临沂大学 基于生成对抗网络的单通道煤岩图像增强方法
CN115661001A (zh) * 2022-12-14 2023-01-31 临沂大学 基于生成对抗网络的单通道煤岩图像增强方法
CN115797163A (zh) * 2023-02-13 2023-03-14 中国人民解放军火箭军工程大学 一种基于遥感影像的目标数据跨域反演增广方法
CN116012258A (zh) * 2023-02-14 2023-04-25 山东大学 一种基于循环生成对抗网络的图像和谐化方法
CN116012258B (zh) * 2023-02-14 2023-10-13 山东大学 一种基于循环生成对抗网络的图像和谐化方法
CN116310348A (zh) * 2023-05-18 2023-06-23 安徽农业大学 一种基于cgan的语义分割数据集增广的方法
CN116310348B (zh) * 2023-05-18 2023-08-22 安徽农业大学 一种基于cgan的语义分割数据集增广的方法
CN117409008A (zh) * 2023-12-15 2024-01-16 华东交通大学 一种设备图像生成方法
CN117409008B (zh) * 2023-12-15 2024-03-22 华东交通大学 一种设备图像生成方法

Also Published As

Publication number Publication date
CN112396110B (zh) 2024-02-02
CN112396110A (zh) 2021-02-23

Similar Documents

Publication Publication Date Title
WO2022105308A1 (zh) 一种基于生成对抗级联网络增广图像的方法
CN109035149B (zh) 一种基于深度学习的车牌图像去运动模糊方法
CN112200750B (zh) 一种超声图像去噪模型建立方法及超声图像去噪方法
CN111325236B (zh) 一种基于卷积神经网络的超声图像分类方法
CN110378844B (zh) 基于循环多尺度生成对抗网络的图像盲去运动模糊方法
CN106204467B (zh) 一种基于级联残差神经网络的图像去噪方法
Tran et al. GAN-based noise model for denoising real images
CN110889895B (zh) 一种融合单帧重建网络的人脸视频超分辨率重建方法
CN111861906B (zh) 一种路面裂缝图像虚拟增广模型建立及图像虚拟增广方法
CN111260591B (zh) 一种基于注意力机制的图像自适应去噪方法
CN110689482A (zh) 一种基于有监督逐像素生成对抗网络的人脸超分辨率方法
CN112487914B (zh) 一种基于深度卷积生成对抗网络的ecg降噪方法
CN113723295B (zh) 一种基于图像域频域双流网络的人脸伪造检测方法
CN111899168B (zh) 一种基于特征增强的遥感影像超分辨率重建方法和系统
CN111932461A (zh) 一种基于卷积神经网络的自学习图像超分辨率重建方法及系统
CN112818969A (zh) 一种基于知识蒸馏的人脸姿态估计方法及系统
CN107483473A (zh) 一种云环境的低速拒绝服务攻击数据流检测方法
Jin et al. Defect identification of adhesive structure based on DCGAN and YOLOv5
CN113592715A (zh) 一种面向小样本图像集的超分辨率图像重构方法
CN116895030A (zh) 基于目标检测算法和注意力机制的绝缘子检测方法
CN109003247B (zh) 彩色图像混合噪声的去除方法
CN114283058A (zh) 基于对抗网络和最大互信息优化的图像超分辨率重建方法
CN112381845A (zh) 岩心图像生成方法、模型训练方法及装置
CN114821174B (zh) 一种基于内容感知的输电线路航拍图像数据清洗方法
Rao et al. Various Degradation: Dual Cross-Refinement Transformer For Blind Sonar Image Super-Resolution

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21893460

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21893460

Country of ref document: EP

Kind code of ref document: A1