CN115690487A - Small sample image generation method - Google Patents

Abstract

The invention discloses a small-sample image generation method, which is used for image generation in a scene with limited data. The small-sample image generation method provided by the present invention includes: randomly sampling from a dynamic Gaussian mixture distribution to obtain a dynamic Gaussian mixture hidden coding; The intermediate features are enhanced, and the intermediate features are obtained from the dynamic Gaussian mixture hidden coding mapping by the generating network, and the enhanced intermediate features are input into the generating network to obtain a generated image set; the generated image set and The real image set is input into the discrimination network to obtain the image discrimination result of the generated image set and the real image set; according to the image discrimination result and the target optimization function of the generation network and the discrimination network to update the The generating network and the discriminant network are used to obtain the updated generating network and discriminant network.

Description

A small sample image generation method

technical field

The invention belongs to the field of computer vision, and mainly relates to the problem of image generation in small sample scenes; it is mainly applied to the fields of image editing, generation, expansion and enhancement, and the like.

Background technique

With the continuous development of deep learning, it has made remarkable progress in the field of computer vision and has been applied in various fields. Among them, the deep image generation model, which uses deep networks to learn and understand the content and distribution of pictures, and learn to generate real pictures similar to real pictures, is a hot issue in the field of computer vision. The deep image generation model is the basis for image inpainting, editing, super-resolution and other tasks, and can be applied to various fields such as film and television media and creative design. However, the training of deep image generation models usually requires a large amount of data and computational costs, which greatly limits the application of generative models in fields with only a few images, such as medical imaging and celebrity paintings. Applying generative models to small sample scenarios is a direction of great application and research significance. It can not only realize data expansion in small sample scenarios through generated data, but also use generated data to assist small sample classification, segmentation and other issues.

When given a small number of images for training, generative models often overfit and simply memorize the training data, failing to produce realistic and diverse images. In order to improve the authenticity and diversity of generated images in small-sample scenes, researchers have proposed various methods to alleviate the model overfitting problem. A direct method is to use the idea of transfer learning, assuming that there is a source domain that is similar to the small sample data and has a large amount of data, first pre-train the source domain, and then transfer the knowledge in the source domain to the small sample target domain to improve Variety and authenticity of small sample generation. However, there are two problems with this type of method: first, the pre-training of the source domain still requires a lot of calculation and acquisition costs; second, when there is a certain deviation between the source domain and the small-sample target domain, it will reduce the small-sample target domain performance.

Another type of small sample generation method uses data enhancement technology to flip and translate small sample data to achieve data expansion and increase the available training data for the model. This type of method also has two problems: first, flipping and translating the original image may change the distribution of the original data, thereby misleading the generative model to produce an unreasonable generated image; second, the augmentation of the sample is essentially still in the Processing the same batch of data will not change the internal structure, so the model is still prone to overfitting.

Contents of the invention

Different from the existing small-sample image generation methods, the present invention starts from the data prior and the essence of the data, and based on the assumption that more complex prior information can provide the model with more editable attributes, and designs Gaussian dynamic mixed latent space coding as the generation method. The input signal of the model provides a more diverse mixture of Gaussian hidden codes to the generative model. At the same time, in order to further ensure the diversity and authenticity of generated pictures, the present invention designs a mixed attention enhancement module for content and layout of intermediate features in the generation process to ensure the rationality and integrity of local content and global layout. By fusing the above two modules, the present invention constructs a small-sample image generation method, which achieves excellent results on small-sample data from different fields including comics and real photos.

The purpose of the present invention is to provide a small-sample image generation method to improve the fidelity and richness of generated pictures in a small-sample scene.

The invention is a small-sample image generation method, which provides more variable and editable attributes for the generation model by introducing dynamic mixed Gaussian hidden coding, and solves the problem of insufficient diversity in the existing small-sample generation methods. At the same time, in order to further improve the authenticity of generated pictures, the present invention proposes a mixed attention mechanism to enhance the global layout and local content of intermediate features in the generation process, effectively retaining the key information of intermediate features. By combining the above methods, the present invention effectively improves the diversity and authenticity of small-sample image generation, and alleviates the problems of model training instability and over-fitting in small-sample scenes.

In order to describe the content of the present invention conveniently, definitions of some commonly used terms are given first.

Definition 1: Generative Adversarial Networks (Generative Adversarial Networks): Generative Adversarial Networks are the most commonly used and most widely used deep generation models. (Generator, G) and a discriminant network (Discriminator, D), where the generator network G maps the hidden code (Latent Code) obtained from a specific distribution sampling into a generated picture, and the generated picture is sent to the discriminant network together with the real picture In D, the discriminant network learns to distinguish the generated pictures from the real pictures, and the training objective function of the generative confrontation network is:

In the above formula, I _real represents the real data distribution, G(z) represents the generated data, and log() represents the logarithmic loss. D and G represent the discriminative network and the generative network, respectively. The discriminant network maximizes the classification loss of generated data and real data, and the generative network minimizes the classification loss of the discriminant network for generated data. The two compete with each other and finally reach an equilibrium state. In the equilibrium state, the generation network has the ability to generate sufficiently realistic pictures, and the discriminant network cannot distinguish whether the picture is real or generated.

Definition 2: Latent Code: The latent code is the input of the generator network G, usually a fixed-length vector randomly sampled from a specific distribution (such as Gaussian distribution, uniform distribution, etc.), denoted by z. The generative network D learns to map the latent code z to the generated image.

Definition 3: Attention Mechanism: Inspired by the human visual attention mechanism, the attention mechanism learns the content that should be more concerned in the image and increases the weight of the corresponding part of the content.

Definition 4: Gaussian Mixture Distribution (Gaussian Mixture Distribution): Gaussian Mixture Distribution represents a distribution composed of N Gaussian distributions. In order to provide more editable and variable attributes for the generation network of the present invention, the present invention uses Gaussian Mixture The hidden codes are randomly sampled from the distribution as the input of the generative network, instead of sampling the hidden codes from a single Gaussian distribution as the input of the generative network as in the existing method.

为第i个高斯分布的权重，(μ_i,∑_i)分别表示第i个高斯分布的均值和方差，z表示混合高斯分布。In the above formula,

is the weight of the i-th Gaussian distribution, (μ _i , ∑ _i ) represent the mean and variance of the i-th Gaussian distribution respectively, and z represents the mixed Gaussian distribution.

Definition 5: Reparameterization Trick (Reparameterization Trick): Since the hidden coding obtained from N Gaussian distribution sampling will cause the neural network to become non-differentiable, the present invention performs reparameterization processing on the Gaussian mixture distribution, and first samples from one of the Gaussian distributions Get δ, and then flatten and reparameterize to get z:

z＝u _i +σ _i δ

In the above formula, u _i and σ _i are learnable parameters of the network, and δ is randomly sampled from a Gaussian distribution with a mean of 0 and a variance of 1, that is, δ∈N(0,1).

In order to further provide more variable and editable attributes for the generation network, the present invention further introduces a dynamic regulation factor λ, which can dynamically regulate the Gaussian component of the hidden code:

z＝λu _i +(1-λ)σ _i δ

In the above formula, λ is a dynamic control factor, u _i and σ _i are network learnable parameters, and δ is randomly sampled from a Gaussian distribution with a mean of 0 and a variance of 1, that is, δ∈N(0,1). z represents the obtained dynamic Gaussian mixture hidden coding.

Definition 6: Sigmoid activation function: The activation function changes the input of the model to obtain a new output through nonlinear changes, and its formalization is defined as:

In the above formula, e represents the natural index, z is the input of the activation function, and g(z) is the Sigmoid activation function.

Definition 7: Convolution: Convolution is a technique for processing images based on the spatial dependence of input image pixels.

Definition 8: Pooling: Pooling reduces the dimensionality of the input feature map according to certain rules, and the selected rules include maximum pooling and average pooling.

A method for generating a small sample image according to the present invention is characterized in that the method for generating a small sample image includes:

Obtaining a dynamic Gaussian mixture hidden code by random sampling from a dynamic Gaussian mixture distribution, the dynamic Gaussian mixture distribution being a Gaussian mixture distribution introducing a dynamic control factor;

The dynamic Gaussian mixture hidden coding is input to the generation network, and the intermediate features of the generation network are enhanced through a mixed attention mechanism, and the intermediate characteristics are obtained by mapping the dynamic Gaussian mixture hidden coding by the generation network, and the enhanced The final intermediate features are input into the generation network to obtain a set of generated images;

Inputting the set of generated images and the set of real images into a discriminant network to obtain an image discrimination result for the set of generated images and the set of real images;

Updating the generation network and the discrimination network according to the image discrimination result and the target optimization functions of the generation network and the discrimination network to obtain updated generation networks and discrimination networks.

A small-sample image generation method described in the present invention helps to solve the problems of over-fitting and model collapse caused by limited data in a small-sample scene, and improves the authenticity and diversity of generated images.

A small-sample image generation method according to the present invention is characterized in that the dynamic Gaussian mixture distribution conforms to the following correspondence:

z＝λu _i +(1-λ)σ _i δ

Among them, z is the dynamic Gaussian mixture distribution, λ is a dynamic control factor, which can adjust the components of the Gaussian distribution in the dynamic Gaussian mixture hidden coding, u _i and σ _i are network learnable parameters, and δ is from the mean value of 0 to the variance of 1 Obtained by random sampling in the Gaussian distribution of , that is, δ∈N(0,1).

A small-sample image generation method according to the present invention is characterized in that the hybrid attention mechanism includes a spatial attention mechanism and a channel attention mechanism. The mixed attention mechanism enhancement enhances the intermediate features of the generation network, and the intermediate features are obtained by mapping the dynamic Gaussian mixture hidden coding by the generation network, which includes the global layout and local content information of the generated picture. Using the mixed attention Force augmentation helps improve the realism of generated images.

和

分别代表使用平均池化和最大池化后得到的所述特征图。接下来，对两个所述特征图进行连接并通过卷积操作得到所述空间注意力的所述特征图。所述空间注意力机制形式化描述为：A small-sample image generation method according to the present invention is characterized in that the spatial attention mechanism focuses on which part of the feature is the most important, and enhances this part of the content. The spatial attention mechanism first uses the pooling operation to aggregate channel information to obtain two 2D feature maps:

and

represent the feature maps obtained after using average pooling and max pooling, respectively. Next, the two feature maps are concatenated and the feature map of the spatial attention is obtained through a convolution operation. The formal description of the spatial attention mechanism is:

和

为所述平均池化和所述最大池化后得到的所述特征图。所述空间注意力关注于全局布局信息，增强全局布局信息有利于生成图像的整体合理性和真实性。Where σ represents the Sigmoid activation function, AvgPool and MaxPool represent the average pooling and the maximum pooling respectively, and f ^7×7 represents a convolution operation with a convolution kernel size of 7×7. F is the feature map,

and

is the feature map obtained after the average pooling and the maximum pooling. The spatial attention focuses on the global layout information, and enhancing the global layout information is conducive to the overall rationality and authenticity of the generated image.

和

然后利用一个网络产生所述通道注意力的所述特征图M_c∈R^c×1×1，这个网络是具有一个隐层的多层感知机。所述通道注意力机制形式化描述为：A small-sample image generation method according to the present invention is characterized in that the channel attention mechanism focuses on what content in the feature map is worthy of attention, and in order to calculate the feature map of the channel attention, first use the average Pooling and maximum pooling squeeze spatial information to obtain two 2D feature maps:

and

The feature map M _c ∈ R ^c×1×1 of the channel attention is then generated using a network, which is a multi-layer perceptron with one hidden layer. The channel attention mechanism is formally described as:

和

为使用所述平均池化和最大池化后得到的所述特征图。所述通道注意力关注于局部内容信息，增强局部内容信息有利于生成图像的局部真实性和细节逼真性。Wherein σ represents the Sigmoid activation function, W ₁ and W ₀ are shared parameters, AvgPool and MaxPool represent the average pooling and the maximum pooling respectively, and MLP is a multi-layer perceptron.

and

is the feature map obtained after using the average pooling and maximum pooling. The channel attention focuses on the local content information, and enhancing the local content information is conducive to the local authenticity and detail fidelity of the generated image.

A small-sample image generation method according to the present invention is characterized in that, according to the image discrimination result, the generation network and the discrimination network are updated according to the target optimization function of the generation network and the discrimination network parameters. include:

Bringing the image discrimination result into the target optimization function of the discrimination network, updating the parameters of the discrimination network, wherein, the target optimization function of the discrimination network conforms to the following corresponding relationship:

Where E represents expectation, I _real is the real image set, z is the dynamic Gaussian mixture distribution, G(z) is the generated image set, D(x) is the image discrimination result, and min() represents the minimum , L _recons is the reconstruction loss, and the reconstruction loss helps to improve the ability of the discriminant network to extract features, thereby improving the discriminative ability of the discriminant network.

Bringing the image discrimination result into the target optimization function of the generation network, updating the parameters of the generation network, wherein, the target optimization function of the generation network conforms to the following correspondence:

L _G ＝-E _x～G(z) [D(x)]

Where E represents expectation, z is the dynamic Gaussian mixture distribution, x˜G(z) is the set of generated images, and D(x) is the image discrimination result.

A small-sample image generation method according to the present invention is characterized in that, after updating the parameters of the generation network and the discrimination network according to the target optimization function of the generation network and the discrimination network, the The method also includes identifying a network based on the updated generative network used to generate image data for at least one of data augmentation, classification, and segmentation.

A small-sample image generation method according to the present invention is characterized in that it includes a generator and a discriminator, the generator is coupled to the generation network, the discriminator is coupled to the discrimination network, and the generation method Programs for implementing the generator and discriminator.

The beneficial effects of the present invention are: the small-sample image generation method designed by the present invention takes the dynamic mixed Gaussian hidden coding as input, provides more editable and more variable attributes for the generation network, and improves the diversity of generated samples; The mixed attention mechanism is used to enhance the local content and global layout of the intermediate features in the generation process to improve the authenticity of the generated samples. The combination of the two alleviates the over-fitting problem of the generative model, and can still generate sufficiently realistic and diverse pictures in small sample scenarios. The present invention is not limited to a specific generative model, and can be adaptively nested into other models to help improve the diversity and authenticity of generated samples, and avoid problems such as mode collapse that often occur in generative models. The generated pictures can also be used in image classification, segmentation and other fields.

Description of drawings

Fig. 1 is an overall training flow chart provided by the embodiment of the present invention.

Fig. 2 is an overall frame diagram provided by an embodiment of the present invention.

Fig. 3 is a diagram of a hybrid attention mechanism provided by an embodiment of the present invention.

Fig. 4 is an effect diagram of the generation of artistic painting scenery and real animal photo data sets provided by the embodiment of the present invention.

Fig. 5 is an effect diagram of generating an animation face and a real face photo data set according to an embodiment of the present invention.

Detailed ways

The implementation of the present invention will be described in detail below with reference to the illustrations. In order to clarify the implementation process of the invention, the practical details of the system will also be described in detail. However, these practical details do not limit the invention to the described embodiments.

The invention is a small-sample image generation method, which uses dynamic Gaussian mixture hidden coding as the input of the generation network to provide richer prior information and more editable attribute information for the generation network; the generation network maps the hidden coding to generate Image, in the process of generating the intermediate image generated by the network, the intermediate feature representation contains the local content and global layout information of the final generated image, using the mixed attention mechanism to enhance the content and layout information of the intermediate representation, and finally generate the image; will generate The picture and the real picture are input into the discriminant model, and the discriminant model needs to identify whether the given picture is generated or real; the generative network and the discriminant network are updated through the discriminative loss, and the generative network must learn to generate pictures that are as close to the real distribution as possible. The discriminant network should distinguish the real picture from the generated picture as much as possible, and the two will play games with each other, get better during continuous training, and finally reach an equilibrium state.

The present invention learns to generate pictures with authenticity and diversity on a small sample image data set based on the generation network and the discrimination network. See Figure 1 for the training process and Figure 2 for the overall framework.

Concrete execution process of the present invention is as follows:

Step 1: Parameter initialization

Initialize training image size D, training set P, Batch Size, number of training iterations T, random initialization of generation network G and discriminant network D;

Step 2: Sampling Dynamic Gaussian Mixture Hidden Coding and Dataset Samples

Randomly sample m hidden codes {z ₁ ,…,z _m } from the dynamic Gaussian mixture distribution, randomly sample m original training pictures {I ₁ ,…,I _m } from the training set P, and the dynamic Gaussian mixture distribution is:

z=λu _i +(1-λ)σ _i δ,δ∈N(0,1)

Among them, λ is a dynamic control factor, which can dynamically adjust the components of the Gaussian distribution in the hybrid hidden coding, and δ is a vector randomly sampled from a Gaussian distribution with a mean of 0 and a variance of 1.

Step 3: Preprocess m original images, horizontally flip, randomly crop and standardize the original images, and represent the data in the form of tensors;

Step 4: Input the hidden code into the generation network. In the middle process of generation, remove the intermediate feature representation, use the mixed attention mechanism to enhance the content and layout of the intermediate feature representation, and the enhanced feature representation will continue to be used in the generation network. , get m generated pictures and process them into the same format as the training pictures {G(z ₁ ),…,G(z _m )};

The mixed attention mechanism described in step 4 includes a spatial attention mechanism and a channel attention mechanism, focusing on the local content and all layout information of the feature map. The process is shown in Figure 3.

Step 5: Input m generated pictures {G(z ₁ ),…,G(z _m )} and m real pictures {I ₁ ,…,I _m } into the discriminant network, and the label of the real picture is “real ", the label of the generated picture is "fake", and let the discriminant network make the discriminative;

Step 6: Train the discriminative network

The discriminative probability of the discriminant network is improved by means of the minimum target loss function, and the gradient descent is used for backpropagation on the loss function of the discriminant network to update the discriminant network parameters;

The loss function of the discriminative network is defined as:

Where E represents expectation, I _real is the real training sample, z is the dynamic Gaussian mixture distribution, G(z) is the generated sample, min() represents the minimization, D(x) is the image discrimination result, L _recons is the reconstruction Loss, the reconstruction loss helps to improve the ability of the discriminant network to extract features, thereby improving the discriminative ability.

Step 7: Train the generative network

The generation network continuously generates pictures under the guidance of the discriminant network. The generated pictures need to be as similar as possible to the real picture to confuse the discriminant network. By minimizing the target loss function of the generative network, the probability of misjudgment by the discriminant network is increased, and the gradient is used to Descent performs backpropagation on the generated network, and updates the parameters of the generated network;

The loss function of the generative network is defined as:

L _G ＝-E _x～G(z) [D(G(z))]

Where E represents expectation, z is the dynamic Gaussian mixture distribution, x˜G(z) is the set of generated images, and D(x) is the image discrimination result.

Step 8: Check the number of iterations. The total number of iterations set by the present invention is 50,000 times. Step 2-Step 7 is repeatedly executed until the termination condition is reached, and the model parameters are saved once every 10,000 iterations. Finally, 5 models are obtained, and the saved model is used It can read and generate network parameters, and generate pictures for visual comparison and quantitative index comparison. The generated pictures can also be used for data enhancement to help improve tasks such as classification and segmentation.

experimental design

Experimental dataset

The experimental data set is selected from small sample image data sets of different styles including animation, painting, face, landscape, etc., including 256*256*3, 512*512*3 and 1024*1024*3 different resolutions, and all data sets There are no more than 1000 pictures, and the data is extremely limited. The detailed introduction of the data is shown in Table 1. It is a big challenge, but also has strong application and research significance.

Table 1 Introduction of experimental data set

Comparison Algorithms

The present invention is aimed at image generation in a small-sample scene, and the comparison algorithm includes the current best methods StyleGAN2, DiffAug, ADA, and FastGAN also in a limited-sample scene.

Evaluation index

A commonly used indicator for evaluating the authenticity and diversity of generated images is FID. FID calculates the distance between the real picture and the generated picture. Referring to the common settings, select the real training picture as the reference picture, generate 5000 pictures, and calculate the distribution distance between the two. The smaller the value, the closer the generated picture is to the real picture, that is Say the better the performance.

The calculation formula of FID is:

FID＝||μ _r -μ _g || ² +Tr(∑ _r +∑ _g -2(∑ _r ∑ _g ) ^1/2 )

Among them, μ _r and μ _g represent the feature mean of the real image and the generated image, respectively, ∑ _r and ∑ _g represent the covariance matrix of the real image and the generated image, respectively, Tr represents the trace, and ||·|| ² represents the two-norm.

Experimental results

Table 2 The experimental results of the present invention and the comparison method on the 256*256*3 data set

Table 3 The experimental results of the present invention and the comparison method on the 512*512*3 data set

Datasets Anime Face ArtPainting Moongate flat Fauvism StyleGAN2 152.73 74.56 288.25 285.61 181.91 Diff Aug 135.85 49.25 136.12 310.14 223.58 ADA 59.67 46.38 149.06 248.46 201.99 FastGAN 59.38 45.08 122.29 240.24 182.14 Ours 53.36 44.50 112.14 200.99 176.43

Table 4 The experimental results of the present invention and the comparison method on the 1024*1024*3 data set

Datasets Pokémon Skulls Shells FFHQ Flowers StyleGAN2 190.23 127.98 241.37 - 45.23 Diff Aug 62.73 124.23 151.94 48.88 37.09 ADA 66.41 97.05 136.52 40.63 27.36 FastGAN 57.19 130.05 155.47 47.78 25.66 Ours 47.04 99.02 134.20 44.01 24.06

Tables 2, 3, and 4 show the experimental results of the method of the present invention and the comparison method on data sets of different resolutions. It can be seen that in the case of a limited sample size, the present invention can generate images with higher authenticity and diversity. The picture proves the effectiveness and superiority of the present invention as a small sample image generation.

visual analysis

In order to better investigate the diversity and authenticity of the effects generated by the present invention on small-sample image datasets, for different resolutions, the visualization results are generated and sorted out, see Figure 4 and Figure 5. It can be seen from the visualization effect in the figure that the pictures produced by the present invention are relatively close to the real pictures, and have achieved good results on data sets with different resolutions. The pictures produced by the present invention are excellent in terms of local content and global layout. quite reasonable.

To sum up, the small sample image generation method proposed by the present invention can significantly improve the diversity and authenticity of the generated pictures in the small sample scene, and the effectiveness and practicability of the system of the present invention are verified from the analysis of quantitative and qualitative results . At the same time, the pictures generated by the present invention can be used for a wide range of tasks in data-limited scenarios, including data enhancement, classification, segmentation, and the like. In addition, the present invention also provides a reference for other related issues in the field. The principles and ideas of the present invention can be extended to other related application scenarios, which has good reference and reference significance, and also provides a very broad application prospect.

The above descriptions are specific implementations of the present invention, and the present invention is not limited to the examples. For those skilled in the art, the present invention can be adapted to various models, and can also be adapted according to specific tasks. Matching adjustments and changes. All modifications, substitutions and improvements made within the principle scope of the present invention shall be included in the claims of the present invention.

Claims (8)

1. A small sample image generation method, comprising:

randomly sampling from dynamic Gaussian mixture distribution to obtain dynamic Gaussian mixture hidden codes, wherein the dynamic Gaussian mixture distribution is the Gaussian mixture distribution with dynamic regulation factors introduced;

inputting the dynamic Gaussian mixture hidden code into a generation network, enhancing the intermediate feature of the generation network through a mixed attention mechanism, wherein the intermediate feature is obtained by mapping the dynamic Gaussian mixture hidden code by the generation network, and the enhanced intermediate feature is input into the generation network to obtain a generated image set;

inputting the generated image set and the real image set into a discrimination network to obtain image discrimination results of the generated image set and the real image set;

and updating the generating network and the judging network according to the image judging result and the target optimization functions of the generating network and the judging network to obtain the updated generating network and the updated judging network.

2. The small sample image generation method according to claim 1, wherein the dynamic gaussian mixture distribution conforms to the following correspondence:

z＝λu _i +(1-λ)σ _i δ

wherein z is the dynamic Gaussian mixture distribution, lambda is a dynamic regulation factor, the components of the Gaussian distribution in the dynamic Gaussian mixture distribution implicit code can be adjusted, and u is _i And σ _i For the network learnable parameter, δ is a vector sampled randomly from a gaussian distribution with mean 0 and variance 1, i.e., δ ∈ N (0,1).

3. The small sample image generation method of claim 1, wherein the hybrid attention mechanism comprises a spatial attention mechanism and a channel attention mechanism.

4. The spatial attention mechanism of claim 3, wherein the spatial attention mechanism focuses on and enhances what portion of the content of a feature is most important. The spatial attention mechanism first aggregates channel information using pooling to obtain two 2D feature maps:

and

representing the profiles obtained using average pooling and maximum pooling, respectively. Next, the two feature maps are connectedAnd obtaining the feature map of the spatial attention through convolution operation. The spatial attention mechanism is formally described as:

where σ denotes the activation function, avgPool and MaxPool denote the mean pooling and the maximum pooling, respectively, f ^{7× 7} f ^7×7 Represents the convolution operation with a convolution kernel of size 7 x 7, F being the signature,

and

and obtaining the characteristic diagram after the average pooling and the maximum pooling.

5. The channel attention mechanism of claim 3, wherein it is interesting to look at what is in a feature map, and to compute the feature map of the channel attention, spatial information is first squeezed using average pooling and maximum pooling, resulting in two 2D feature maps:

and

then, a network is used to generate the feature map M of the channel attention _c ∈R ^c×1×1 The network is a multi-layered perceptron with a hidden layer. The channel attention mechanism is formally described as:

where σ denotes a Sigmoid activation function, W ₁ And W ₀ For sharing parameters, avgPool and MAxPool represent average pooling and maximum pooling, respectively, MLP is a multilayer perceptron,

and

the feature maps obtained after using the average pooling and the maximum pooling.

6. The method according to any one of claims 1 to 5, wherein the parameters of the generation network and the discrimination network are updated according to an objective optimization function of the generation network and the discrimination network according to the image discrimination result. The method comprises the following steps:

substituting the image discrimination result into the target optimization function of the discrimination network, and updating the parameters of the discrimination network, wherein the target optimization function of the discrimination network conforms to the following corresponding relation:

wherein E represents desire, I _real For the real image set, z is the dynamic Gaussian mixture distribution, G (z) is the generated image set, D (x) is the image discrimination result, min () represents minimization, L _recons In order to reconstruct the loss, the reconstruction loss helps to improve the capability of the discrimination network for extracting features, thereby improving the discrimination capability of the discrimination network.

Substituting the image discrimination result into the target optimization function of the generated network, and updating the parameters of the generated network, wherein the target optimization function of the generated network conforms to the following corresponding relation:

L _G ＝-E _x～G(z) [D(x)]

where E represents expectation, z is the dynamic Gaussian mixture distribution, x-G (z) are the generated image set, and D (x) is the image discrimination result.

7. The method of any one of claims 1 to 6, wherein after said updating parameters of said generator network and said discriminant network according to an objective optimization function of said generator network and said discriminant network, said method further comprises:

and judging a network according to the updated generating network, wherein the updated generating network is used for generating image data for at least one of data enhancement, classification and segmentation.

8. A small sample image generation method comprising a generator coupled to the generation network and a discriminator coupled to the discrimination network, the generation method being configured to execute a program of the generator and the discriminator so that the generation system performs the method of any one of claims 1 to 7.

Images