AU2017101166A4 - A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks - Google Patents

A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks Download PDF

Info

Publication number
AU2017101166A4
AU2017101166A4 AU2017101166A AU2017101166A AU2017101166A4 AU 2017101166 A4 AU2017101166 A4 AU 2017101166A4 AU 2017101166 A AU2017101166 A AU 2017101166A AU 2017101166 A AU2017101166 A AU 2017101166A AU 2017101166 A4 AU2017101166 A4 AU 2017101166A4
Authority
AU
Australia
Prior art keywords
image
loss
style
cgan
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
AU2017101166A
Inventor
Haodong Lai
Qinlai Li
Yutao LIU
Haolun Wu
Wentao Zhou
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to AU2017101166A priority Critical patent/AU2017101166A4/en
Application granted granted Critical
Publication of AU2017101166A4 publication Critical patent/AU2017101166A4/en
Ceased legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The present invention discloses a method for achieving image style transfer in real time by using conditional Generative Adversarial Networks(cGAN). By using these networks, the invention achieves a good performance at high-level image information. The cGAN networks not only learn the mapping from input images to output images, but also learn a loss function to train this mapping. The main steps include: using a traditional way to generate training datasets to get input and target pairs of images; puttingimages into the encoder of cGAN to extract the features and decoding them by using a "U-Net" architecture which is an advanced convolutional network; optimizing the values of generator and discriminator simultaneously by using a gradient descent approach based on the established loss functions and ultimately getting our results. The cGAN makes it possible to applythe same generic approach to problems that traditionallywould require very different loss formulations.In addition to the advantage of the fast speed, this invention is effective especially for dealing with high level image features and synthesis which can have more explicit results. Training Generasets epoch= epoch(max) input contend of the figure ) No so sy while epoch !ck0 s tran ing Yes encoder Genratr: feature U-net decoder (U-net) style figure style pic(fake) style pi ( real) discriminator update the parameter update the parameter of G to reduce loss of D to reduce loss (loss=lp+1GAN) (loss=ID3) epoch-L Fig. 1.

Description

TITLE A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks
FIELD OF THE INVENTION
The present invention generally relates to an image-to-image processing technique for style transfer in a fast way, especially based on a method by using conditional Generative Adversarial Networks (cGAN).
BACKGROUND OF THE INVENTION
Transferring the style from one image onto another can be regarded as a problem of texture transfer, which is of practical significance. In daily life, human beings usually want to transfer their photographs and paintings to specific styles by post editing. Professional image editing requires a great artistic talent and rich experience, which is never easy for ordinary people to do so.
Most previous transfer algorithms rely on non-parametric methods to synthesize photo-realistic natural textures by resampling the pixels of a given source texture while using different ways to preserve the structure of the target image. More recently, plenty of image synthesis methods based on deep neural networks have emerged. The related reference references are: [1JN. Ashikhmin. Fast texture transfer. IEEE Computer Graphics and Applications, 23(4):38-43, July 2003.
[2] A. A. Efros and W. T. Freeman. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 341-346. ACM, 2001.
[3] Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning withdeep conditional generative adversarial networks. CoRR abs/1511.06434 (2015).
Although these methods achieve remarkable results, there are some notable limitations. The non-parametric methods merely focus on the low-level features, so that they cannot perform well in generating an ideal image when considering advanced high-level style features. What’s more, when using the methods on deep neural networks, the outputs of networks are difficult to be controlled because of too much freedom in the model. Therefore, a basic prerequisite is to find an independent model to synthesize detail features of images beyond what is contained in the input. The model should solve the above problems by learning both the pixel elements and the high-level style features.
SUMMARY OF THE INVENTION
The objects of present invention are to put forward a method for achieving an image-to-image style transfer in a fast way to avoid the above problems. The present invention disclosed an approach based on the conditional Generative Adversarial Networks(cGAN).
Image-to-image translation problems are often formulated as per-pixel classification or regression. These formulations treat the output space as “unstructured” in the sense that each output pixel is considered conditionally independent from all others given the input image. Conditional GANs instead learn a structured loss. Structured losses penalize the joint configuration of the output. Besides, Previous works have conditioned GANs on natural images. The image-conditional models have tackled inpainting, image prediction from a normal map, image manipulation guided by user constraints, future frame prediction, future state prediction, product photo generation, and style transfer. The related references are as follows: [4] C. Li and M. Wand. Combining markov random fields and conditional neural networks for image synthesis. CVPR, 2016.
[5] C. Li and M. Wand. Precomputed real-time texture synthesis with markovian generative adversarial networks. ECCV, 2016.
[6JJ.-Y. Zhu, P. Krahenbuhl, E. Shechtman, and A. A.
Efros. "Generative visual manipulation on the natural image manifold. Ln ECCV, 2016.
The conditional GAN in this invention is different in that the loss is learned, and can, in theory, penalize any possible structure that differs between output and target. The method also differs from other works in some choices. Unlike previous works, this invention uses a “U-Net” architecture, and for a discriminator it uses a conditional “PatchGAN” classifier. GANs are generative models that learn a mapping from random noise vector z to output image T: G: z -> y jn contrast, conditional GANs learn a mapping from observed image x and random noise vector z, to y: G: {x,z} y The generator G is trained to produce outputs that cannot be distinguished from “real” images by an adversarially trained discriminator/), which is trained to do as well as possible at detecting the generator’s “fake”. Therefore, we use a traditional model to train an abundant of pictures and get the style images which are regarded as the “real”, while we input the same training set into the model in this invention and what we get is the “fake”. Our ideal goal is to let the discriminator D cannot distinguish the “real” and the “fake”.
Besides, we provide a Gaussian noise z as an input of generator, in addition to x.
Each time we get an image by using the generator G, we send that output image to discriminator D. The discriminator D will try its best to judge the output is whether “fake” or “real” . If it is evaluated as “fake”, a full 0 matrix will be used as an output. Otherwise, if it is evaluated as “real”, a full 1 matrix will be considered as an output in this step.
The objective of a conditional GAN can be expressed as
Lcgan (G, D) = Exy„pM[ log D(x, y)] +
Ex~Pdm (x\z~px (Z) [log(l - E>(x, G(x>z))]
Then we define Lp as the difference between generate image and the ground truth(the “real” and the “fake”) at pixel level, which can be expressed as Lreal ~ Lfake. More specifically, we choose the matrix norm to formulate it as
Based on the principle of cGAN, G tries to minimize this objective against an adversarial D that tries to maximize it, i.e. G* = arg minc maxn LcGAN (G, D).
Thus, we use Lcgan(G,D) com^inecj wjth LP(G) to train the generator to minimize the loss and our final objective is G' = arg minG maxD LcCiAN (G, D) + XLp (G)
Similarly, the discriminator is simultaneous trained to maximize the loss: ^(^wPoga-^GKz))]
So, we update the weights of the encoder-unit in generator to minimize E(G,D) ancj simultaneously update the weights of the unitin discriminator to maximize V(G> , while x is the input image, y is the target style image and is the generated image.
Many previous solutions to problems in this area have used an encoder-decoder network, such as [7]G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science,313(5786):504-507, 2006. In such a network, the input is passed through a series of layers that progressively downsample, until a bottleneck layer, at which point the process is reversed. Such a network requires that all information flow pass through all the layers, including the bottleneck. For many image translation problems, there is a great deal of low-level information shared between the input and output, and it would be desirable to shuttle this information directly across the net.
To give the generator a means to circumvent the bottleneck for information like this, we add skip connections, following the general shape of a “U-Net”. Specifically, we add skip connections between each layer i and layer n - i, where n is the total number of layers. In this invention, the n equals to 8. Each skip connection simply concatenates all channels at layer i with those at layer n - i.
Consequently, we avoid a massive calculation when transferring each picture for a specific style. Instead, we let our our model learn the implementation way in advance, which means each time we get an input, we can have our results in real time. Additionally, the use of “U-Net” architecture avoids the loss of information in every step of the encoder and decoder and offers more information during the deconvolution. Eventually, we have a good result especially for the semantic features in a high-level and obtain a real-time method to achieve image style transfer.
DESCRIPTION OF DRAWINGS
The following drawings are only for the purpose of description and explanation butnot for limitation, where in:
Fig.l is the flow diagram of the core algorithm of our invention, which shows the specific step in our work step by step;
Fig.2 is the structure of patchGAN discriminator: inputof the original image (yellow block) and the “real” target image(red block), and the “fake” image (blue block);
Fig.3 shows two choices for the architecture of the generator. The“U-Net” is an encoder-decoder with skip connections between mirrored layers in the encoder and decoder stacks;
Fig.4 illustrates the “U-net” architecture (example for 32x32 pixels in the lowest resolution). Each bluebox corresponds to a multi-channel feature map. The number of channels is denotedon top of the box. The x-y-size is provided at the lower left edge of the box. Whiteboxes represent copied feature maps. The arrows denote the differentoperations;
Fig.5 shows the process of training a conditional GAN to predict aerial photos frommaps. The discriminator, D, learns to classify between real andsynthesized pairs. The generator learns to fool the discriminator. Both the generator and discriminator observe an input image.
Fig. 6 shows the results of our invention. The first column lists the original images we used in our model, while the first row shows different styles. Each image at the cross point is our result after transferring the style of the input image(at the left of the row) to the specific style (at the top of the column).
DESCRIPTION OF PREFERRED EMBODIMENT
In order to make the present invention more readily understood, the embodiments of the present invention will be explained in details. The whole process is shown in Fig. 1.
Compared with the traditional way of image style transfer, we used cGAN to make the output more realistic in an efficient way. The generator of cGAN is a symmetric structure. Its encoder and decoder have 8 convolutional and deconvolutional layers respectively.
Step A: We use a traditional way to generate training sets. The related reference is [8]LA Gatys , AS Ecker , M Bethge, Image Style Transfer Using Convolutional Neural Networks, IEEE Conference on Computer Vision & Pattern Recognition , 2016:2414-2423 . We need to get enough images for training the model to generate realistic images, so we generate data sets which contain 1000 images. 800 of them are used to train the model and the rests are used to test. Generating an image needs 30 seconds by using this model.
Step β.-Afterobtaining the content image (input) and transferred image (target) pairs, we need to map the input domain to the output domain. We put the content image into the encoder of cGAN. An image can be regarded as atensor of three dimention, or threematrices since one image consist of three color channel red, blue and green. The convolutional layer is the core building block of a CNN, of which the outputs are feature maps. Parameters of each layer consist of filters and biases. For a certain layer, each feature map is calculated by convolving all feature maps in former layer by its own filter and summed together, then a bias is added. Thus every filter can produce a feature map, and then we stack the series of feature maps along the depth dimension forms the full output volume of the convolution layer. The encoder is a CNN which consists of 8 convolutional layers and for each layer we use convolutional kernels to extract the features of each image channel. The first layer of the encoder can extract low-level features of the image, such as the edges in the image. The subsequent convolutional layers obtain higher level features such as the object parts, then the semantic features. We also need to resize the image to make it smaller, because an original image has many parameters so it is a complex way to get its features. Three shrinking layers are stacked in the middle of the convolutional layers. These three shrinking layers can also be regarded as performing linear combination within the learned features. The traditional way to shrink the image channel is pooling, which is a form of non-linear down-sampling. It divides an input image into a set of non-overlapping rectanglesand for each such sub-region, outputs the maximum. However, this way may cause the loss of important information. Instead of using pooling, we use stride to control the filter convolved across each part of the image, it can learn the downsampling process by itself, it will know the most important feature of each part of the image. It would also reduce the number of layers, so we can get the most useful and sharp features of the image as well as shrink the spatial size. The stride reduces the parameter of the image by shrink the image size, thus the encoder can get the feature vectors faster. For example, when the image passes each convolutional layer, the size of the image can be from 256 *256 to be 128*128, and then to be 64*64. Finally, the image becomes small with many extracted features, which can keep more useful information, and that means we get the feature vectors in the encoder parts.
Step C: Now we have feature vectors and we use decoder to transform those vectors into a new image. It is like a reverse process of encoder and has #deconvolutional layers as well. The deconvolutional block is an upsampling process. It is an algorithm-based process used to reverse the effects of convolution on recorded data. The concept of deconvolution is widely used in the techniques of signal processing and image processing.Eventually the output is an image with different style. That is how the traditional way to transfer the style of the image. However, the input goes through a series of down sampling steps to a lower dimension, then gets some non-linear transformation with a fully connected layer and finally gets up sampling to the present output size. Such a structure may cause information loss when passing through layers. We need to keep the edge as important part to keep the output high quality. So we use the “U-Net” to concatenate layers in encoder to the corresponding layers of decoder. This way can help us keep the edge and other important information of the image, the output would be clearer instead of blurry. The “U-Net” architecture is shown in Fig. 3 and Fig. 4 in detail.
Step Z>:We simultaneously train the generator and the discriminator, consequently the generator has the ability to produce image whose content is consistent with the input image (i.e. the original image whose style needs to be transferred), and at the same time the style is consistent with the target style which is provided by the style images in training set. The traditional way to improve the quality of image is that the generator adjusts its parameter based on the content loss and feature loss, and for every style transformation it adjusts the parameter layer by layer from the output one to the input, and it is a slow and inefficient way. We instead generate the image in a feed-forward way. Firstly, the model is trained by Generator-Discriminator to adjust its parameters to train the model and finally get realistic images, after the finish of training the generator of cGAN can generate image in real time without any adjustment, which is efficient. For the output image, the discriminator of cGAN can identify the loss of the image. The structure of the discriminator is shown in Fig. 2. In image synthesis with GAN, the generator attempts to produce a realistic image from an input random vector to fool the simultaneously adversarial trained discriminator, which needs to distinguish the authentic and fake pairs of images. It is like two-player game, and players improve their ability through training, which means the ability of generator and discriminator are both improved through the training. The goal of the training is the synthesized image generated by G which is indistinguishable from actual distribution by the discriminator D. The process of this part can be illustrated in Fig. 5. The objective (loss) function of general cGAN can be expressed by: min max V(G,D) = Εχ^ΡώΜ(,>j0 [log D(x, y)] +
Ex~pM~p*»U ^1°^1 - D(X’G(x, Z)))\ where the generator G tries to minimize the objective function while an adversarial D tries to maximize it. To understand the difference between the fake and real images better, we defined a loss function which can be expressed by:
^ = ^pLp + XaLGAN
Where Lp is pix loss which refers to thestandard deviation of the pixel between the generated image and the authentic image. LGAN is adversarial loss which is from discriminator. λρ and λα are pre-defined weights for perceptual loss and adversarial loss. If we set both λρ and λα to be 0, then the network reduces to a normal CNN configuration, which aims to minimize only the Euclidean loss between output image and ground truth. If λρ is set to 0, then the network reduces to a normal GAN. If λα set to 0, then the network reduces to the structure. Given an image pair{x,yh} with C channels, width Wand height //(i.e. CxWxH), where x is the input, image and yh is the corresponding ground truth. The Lp can be expressed by:
Given a set of N input images (ground truth or de-rained result) (y, with the corresponding labels /, (7 represents “real”, 0 represents “fake”), the binary cross-entropy loss for the discriminator D is defined as:
L can express the difference between the fake and real image, so it can help G and D train better. After the training process, the generator can produce almost authentic images with transferred style.
Step is:We have got trained parameters from the training model, which can help the model generate transferred images in real time. We create a model used to test and restore the parameters, so the model can generate images directly. We use content image as input and output per image with different style less than Is, and that is much faster than the traditional way which needs 30s to generate an image.
Ultimately, we write ahypertext mark-up language(/i/m/) to show our results. Part of our results are shown in Fig. 6.
There are several key advantages in this invention:
By using the cGAN, we obtain substantial diversity which extract the feature of the input picture more explicitly. And the test results prove that these networks have a better performance than the previous related works particularly in dealing with the high-level features.
Besides, for each style transfer, we let our our model learn the implementation way in advance, which means each time we get an input, we can have our results in real time. Conversely, by using a traditional way, a massive calculation at every time is unavoidable even for transferring the same style.
What’s more, the “U-Net” architecture avoids the loss of information in every step of the encoder and decoder. We use a “U-Net” architecture to splice the two feature map together in order to offer more information during the deconvolution; thus we have a good result especially for the semantic features in a high-level.

Claims (2)

1. This method for real-time image style transfer isbased onconditional Generative Adversarial Networks(cGAN), which includes following steps: using a traditional way to generate training datasets to get input and target pairs of images; puttingimages into the encoder of cGAN to extract the features and decoding them by using a “U-Net” architecture which is an advanced convolutional network; optimizing the values of generator and discriminator simultaneously by using a gradient descent approach based on the established loss functions and ultimately getting our results; a PatchGAN structure is also used in this model for evaluation.
AU2017101166A 2017-08-25 2017-08-25 A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks Ceased AU2017101166A4 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2017101166A AU2017101166A4 (en) 2017-08-25 2017-08-25 A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
AU2017101166A AU2017101166A4 (en) 2017-08-25 2017-08-25 A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks

Publications (1)

Publication Number Publication Date
AU2017101166A4 true AU2017101166A4 (en) 2017-11-02

Family

ID=60163022

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2017101166A Ceased AU2017101166A4 (en) 2017-08-25 2017-08-25 A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks

Country Status (1)

Country Link
AU (1) AU2017101166A4 (en)

Cited By (72)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767384A (en) * 2017-11-03 2018-03-06 电子科技大学 A kind of image, semantic dividing method based on dual training
CN107886510A (en) * 2017-11-27 2018-04-06 杭州电子科技大学 A kind of prostate MRI dividing methods based on three-dimensional full convolutional neural networks
CN107909537A (en) * 2017-11-16 2018-04-13 厦门美图之家科技有限公司 A kind of image processing method and mobile terminal based on convolutional neural networks
CN107944483A (en) * 2017-11-17 2018-04-20 西安电子科技大学 Classification of Multispectral Images method based on binary channels DCGAN and Fusion Features
CN108021131A (en) * 2017-11-28 2018-05-11 王智华 A kind of robot vision recognition methods, device and computer-readable recording medium
CN108038823A (en) * 2017-12-06 2018-05-15 厦门美图之家科技有限公司 Image-type becomes the training method of network model, image-type becomes method and computing device
CN108038821A (en) * 2017-11-20 2018-05-15 河海大学 A kind of image Style Transfer method based on production confrontation network
CN108038860A (en) * 2017-11-30 2018-05-15 杭州电子科技大学 Spine segmentation method based on the full convolutional neural networks of 3D
CN108205659A (en) * 2017-11-30 2018-06-26 深圳市深网视界科技有限公司 Face occluder removes and its method, equipment and the medium of model construction
CN108460720A (en) * 2018-02-01 2018-08-28 华南理工大学 A method of changing image style based on confrontation network model is generated
CN108564166A (en) * 2018-03-22 2018-09-21 南京大学 Based on the semi-supervised feature learning method of the convolutional neural networks with symmetrical parallel link
CN108564611A (en) * 2018-03-09 2018-09-21 天津大学 A kind of monocular image depth estimation method generating confrontation network based on condition
CN108596343A (en) * 2018-03-29 2018-09-28 杭州电子科技大学 A method of it is improved based on feedback regulation and generates confrontation network stabilization
CN108629823A (en) * 2018-04-10 2018-10-09 北京京东尚科信息技术有限公司 The generation method and device of multi-view image
CN108648196A (en) * 2018-03-22 2018-10-12 广州多维魔镜高新科技有限公司 Image partition method and storage medium based on recurrence linking convolutional neural networks
CN108711141A (en) * 2018-05-17 2018-10-26 重庆大学 The motion blur image blind restoration method of network is fought using improved production
CN108734653A (en) * 2018-05-07 2018-11-02 商汤集团有限公司 Image style conversion method and device
CN108875766A (en) * 2017-11-29 2018-11-23 北京旷视科技有限公司 Method, apparatus, system and the computer storage medium of image procossing
CN108876726A (en) * 2017-12-12 2018-11-23 北京旷视科技有限公司 Method, apparatus, system and the computer storage medium of image procossing
CN108922518A (en) * 2018-07-18 2018-11-30 苏州思必驰信息科技有限公司 voice data amplification method and system
CN109131348A (en) * 2018-07-24 2019-01-04 大连理工大学 A kind of intelligent vehicle Driving Decision-making method based on production confrontation network
CN109147810A (en) * 2018-09-30 2019-01-04 百度在线网络技术(北京)有限公司 Establish the method, apparatus, equipment and computer storage medium of speech enhan-cement network
CN109165664A (en) * 2018-07-04 2019-01-08 华南理工大学 A kind of attribute missing data collection completion and prediction technique based on generation confrontation network
CN109308679A (en) * 2018-08-13 2019-02-05 深圳市商汤科技有限公司 A kind of image style conversion side and device, equipment, storage medium
CN109388565A (en) * 2018-09-27 2019-02-26 西安电子科技大学 Software system performance optimization method based on production confrontation network
CN109544545A (en) * 2018-11-30 2019-03-29 济南浪潮高新科技投资发展有限公司 A kind of salt mine intelligent detecting method and system based on convolutional neural networks
CN109583345A (en) * 2018-11-21 2019-04-05 平安科技(深圳)有限公司 Roads recognition method, device, computer installation and computer readable storage medium
CN109671125A (en) * 2018-12-17 2019-04-23 电子科技大学 A kind of GAN network model that height merges and the method for realizing text generation image
CN109685811A (en) * 2018-12-24 2019-04-26 北京大学第三医院 PET/CT hypermetabolism lymph node dividing method based on dual path U-net convolutional neural networks
CN109801228A (en) * 2018-12-18 2019-05-24 合肥阿巴赛信息科技有限公司 A kind of jewelry picture beautification algorithm based on deep learning
CN109829849A (en) * 2019-01-29 2019-05-31 深圳前海达闼云端智能科技有限公司 A kind of generation method of training data, device and terminal
CN110008846A (en) * 2019-03-13 2019-07-12 南京邮电大学 A kind of image processing method
CN110070587A (en) * 2019-03-05 2019-07-30 浙江工商大学 It is a kind of to cascade the pedestrian image generation method that confrontation generates network based on condition
WO2019144855A1 (en) * 2018-01-26 2019-08-01 腾讯科技(深圳)有限公司 Image processing method, storage medium, and computer device
CN110136063A (en) * 2019-05-13 2019-08-16 南京信息工程大学 A kind of single image super resolution ratio reconstruction method generating confrontation network based on condition
CN110211203A (en) * 2019-06-10 2019-09-06 大连民族大学 The method of the Chinese character style of confrontation network is generated based on condition
CN110298844A (en) * 2019-06-17 2019-10-01 艾瑞迈迪科技石家庄有限公司 X-ray contrastographic picture blood vessel segmentation and recognition methods and device
CN110347819A (en) * 2019-06-21 2019-10-18 同济大学 A kind of text snippet generation method based on positive negative sample dual training
CN110533580A (en) * 2019-08-08 2019-12-03 西安交通大学 A kind of image Style Transfer method generating neural network based on confrontation
CN110619315A (en) * 2019-09-24 2019-12-27 重庆紫光华山智安科技有限公司 Training method and device of face recognition model and electronic equipment
CN110634108A (en) * 2019-08-30 2019-12-31 北京工业大学 Composite degraded live webcast video enhancement method based on element-cycle consistency countermeasure network
WO2020024206A1 (en) * 2018-08-02 2020-02-06 深圳先进技术研究院 Dcgan-based parking data repairing method and apparatus, and device and storage medium
CN110796229A (en) * 2018-08-01 2020-02-14 华为技术有限公司 Device and method for realizing convolution operation
CN110826059A (en) * 2019-09-19 2020-02-21 浙江工业大学 Method and device for defending black box attack facing malicious software image format detection model
CN110909754A (en) * 2018-09-14 2020-03-24 哈尔滨工业大学(深圳) Attribute generation countermeasure network and matching clothing generation method based on same
WO2020073758A1 (en) * 2018-10-10 2020-04-16 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for training machine learning modle, apparatus for video style transfer
CN111028306A (en) * 2019-11-06 2020-04-17 杭州电子科技大学 AR2U-Net neural network-based rapid magnetic resonance imaging method
CN111047507A (en) * 2019-11-29 2020-04-21 北京达佳互联信息技术有限公司 Training method of image generation model, image generation method and device
CN111079562A (en) * 2019-11-27 2020-04-28 浙江大学 Multi-stage data generation self-circulation financial invoice text intelligent identification system and method
CN111242953A (en) * 2020-01-17 2020-06-05 陕西师范大学 MR image segmentation method and device based on condition generation countermeasure network
CN111260545A (en) * 2020-01-20 2020-06-09 北京百度网讯科技有限公司 Method and device for generating image
CN111260620A (en) * 2020-01-14 2020-06-09 三峡大学 Image anomaly detection method and device and electronic equipment
CN111275784A (en) * 2020-01-20 2020-06-12 北京百度网讯科技有限公司 Method and device for generating image
CN111340060A (en) * 2018-12-19 2020-06-26 财团法人工业技术研究院 Training method of image generator
CN111402121A (en) * 2020-03-20 2020-07-10 上海眼控科技股份有限公司 Image style conversion method and device, computer equipment and storage medium
CN111583168A (en) * 2020-06-18 2020-08-25 上海眼控科技股份有限公司 Image synthesis method, image synthesis device, computer equipment and storage medium
CN111724299A (en) * 2020-05-21 2020-09-29 同济大学 Super-realistic painting image style migration method based on deep learning
CN111754389A (en) * 2019-03-27 2020-10-09 通用汽车环球科技运作有限责任公司 Semantic preserving style transfer
CN111784592A (en) * 2020-05-19 2020-10-16 知昇(上海)人工智能科技有限公司 Automatic design image generation method based on GAN
CN111784565A (en) * 2020-07-01 2020-10-16 北京字节跳动网络技术有限公司 Image processing method, migration model training method, device, medium and equipment
CN111899251A (en) * 2020-08-06 2020-11-06 中国科学院深圳先进技术研究院 Copy-move type forged image detection method for distinguishing forged source and target area
CN112001122A (en) * 2020-08-26 2020-11-27 合肥工业大学 Non-contact physiological signal measuring method based on end-to-end generation countermeasure network
US10891949B2 (en) 2018-09-10 2021-01-12 Ford Global Technologies, Llc Vehicle language processing
CN112288622A (en) * 2020-10-29 2021-01-29 中山大学 Multi-scale generation countermeasure network-based camouflaged image generation method
CN112731522A (en) * 2020-12-14 2021-04-30 中国地质大学(武汉) Intelligent recognition method, device and equipment for seismic stratum and storage medium
CN113538224A (en) * 2021-09-14 2021-10-22 深圳市安软科技股份有限公司 Image style migration method and device based on generation countermeasure network and related equipment
CN113538216A (en) * 2021-06-16 2021-10-22 电子科技大学 Image style migration method based on attribute decomposition
CN113724132A (en) * 2021-11-03 2021-11-30 浙江宇视科技有限公司 Image style migration processing method and device, electronic equipment and storage medium
CN114863225A (en) * 2022-07-06 2022-08-05 腾讯科技(深圳)有限公司 Image processing model training method, image processing model generation device, image processing equipment and image processing medium
CN115546589A (en) * 2022-11-29 2022-12-30 浙江大学 Image generation method based on graph neural network
CN115588070A (en) * 2022-12-12 2023-01-10 南方科技大学 Three-dimensional image stylized migration method and terminal
CN116630140A (en) * 2023-03-31 2023-08-22 南京信息工程大学 Method, equipment and medium for realizing animation portrait humanization based on condition generation countermeasure network

Cited By (103)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767384A (en) * 2017-11-03 2018-03-06 电子科技大学 A kind of image, semantic dividing method based on dual training
CN107909537B (en) * 2017-11-16 2020-11-06 厦门美图之家科技有限公司 Image processing method based on convolutional neural network and mobile terminal
CN107909537A (en) * 2017-11-16 2018-04-13 厦门美图之家科技有限公司 A kind of image processing method and mobile terminal based on convolutional neural networks
CN107944483A (en) * 2017-11-17 2018-04-20 西安电子科技大学 Classification of Multispectral Images method based on binary channels DCGAN and Fusion Features
CN107944483B (en) * 2017-11-17 2020-02-07 西安电子科技大学 Multispectral image classification method based on dual-channel DCGAN and feature fusion
CN108038821A (en) * 2017-11-20 2018-05-15 河海大学 A kind of image Style Transfer method based on production confrontation network
CN107886510A (en) * 2017-11-27 2018-04-06 杭州电子科技大学 A kind of prostate MRI dividing methods based on three-dimensional full convolutional neural networks
CN108021131A (en) * 2017-11-28 2018-05-11 王智华 A kind of robot vision recognition methods, device and computer-readable recording medium
CN108021131B (en) * 2017-11-28 2021-02-09 王智华 Robot vision identification method and device and computer readable storage medium
CN108875766A (en) * 2017-11-29 2018-11-23 北京旷视科技有限公司 Method, apparatus, system and the computer storage medium of image procossing
CN108875766B (en) * 2017-11-29 2021-08-31 北京旷视科技有限公司 Image processing method, device, system and computer storage medium
CN108205659A (en) * 2017-11-30 2018-06-26 深圳市深网视界科技有限公司 Face occluder removes and its method, equipment and the medium of model construction
CN108038860A (en) * 2017-11-30 2018-05-15 杭州电子科技大学 Spine segmentation method based on the full convolutional neural networks of 3D
CN108038823B (en) * 2017-12-06 2021-07-09 厦门美图之家科技有限公司 Training method of image morphing network model, image morphing method and computing device
CN108038823A (en) * 2017-12-06 2018-05-15 厦门美图之家科技有限公司 Image-type becomes the training method of network model, image-type becomes method and computing device
CN108876726A (en) * 2017-12-12 2018-11-23 北京旷视科技有限公司 Method, apparatus, system and the computer storage medium of image procossing
US11276207B2 (en) 2018-01-26 2022-03-15 Tencent Technology (Shenzhen) Company Limited Image processing method, storage medium, and computer device
WO2019144855A1 (en) * 2018-01-26 2019-08-01 腾讯科技(深圳)有限公司 Image processing method, storage medium, and computer device
CN108460720A (en) * 2018-02-01 2018-08-28 华南理工大学 A method of changing image style based on confrontation network model is generated
CN108564611A (en) * 2018-03-09 2018-09-21 天津大学 A kind of monocular image depth estimation method generating confrontation network based on condition
CN108564166A (en) * 2018-03-22 2018-09-21 南京大学 Based on the semi-supervised feature learning method of the convolutional neural networks with symmetrical parallel link
CN108648196A (en) * 2018-03-22 2018-10-12 广州多维魔镜高新科技有限公司 Image partition method and storage medium based on recurrence linking convolutional neural networks
CN108596343A (en) * 2018-03-29 2018-09-28 杭州电子科技大学 A method of it is improved based on feedback regulation and generates confrontation network stabilization
CN108629823B (en) * 2018-04-10 2022-09-06 北京京东尚科信息技术有限公司 Method and device for generating multi-view image
CN108629823A (en) * 2018-04-10 2018-10-09 北京京东尚科信息技术有限公司 The generation method and device of multi-view image
CN108734653B (en) * 2018-05-07 2022-05-13 商汤集团有限公司 Image style conversion method and device
CN108734653A (en) * 2018-05-07 2018-11-02 商汤集团有限公司 Image style conversion method and device
CN108711141B (en) * 2018-05-17 2022-02-15 重庆大学 Motion blurred image blind restoration method using improved generation type countermeasure network
CN108711141A (en) * 2018-05-17 2018-10-26 重庆大学 The motion blur image blind restoration method of network is fought using improved production
CN109165664B (en) * 2018-07-04 2020-09-22 华南理工大学 Attribute-missing data set completion and prediction method based on generation of countermeasure network
CN109165664A (en) * 2018-07-04 2019-01-08 华南理工大学 A kind of attribute missing data collection completion and prediction technique based on generation confrontation network
CN108922518A (en) * 2018-07-18 2018-11-30 苏州思必驰信息科技有限公司 voice data amplification method and system
CN109131348A (en) * 2018-07-24 2019-01-04 大连理工大学 A kind of intelligent vehicle Driving Decision-making method based on production confrontation network
CN110796229A (en) * 2018-08-01 2020-02-14 华为技术有限公司 Device and method for realizing convolution operation
WO2020024206A1 (en) * 2018-08-02 2020-02-06 深圳先进技术研究院 Dcgan-based parking data repairing method and apparatus, and device and storage medium
CN109308679A (en) * 2018-08-13 2019-02-05 深圳市商汤科技有限公司 A kind of image style conversion side and device, equipment, storage medium
CN109308679B (en) * 2018-08-13 2022-08-30 深圳市商汤科技有限公司 Image style conversion method and device, equipment and storage medium
US10891949B2 (en) 2018-09-10 2021-01-12 Ford Global Technologies, Llc Vehicle language processing
CN110909754B (en) * 2018-09-14 2023-04-07 哈尔滨工业大学(深圳) Attribute generation countermeasure network and matching clothing generation method based on same
CN110909754A (en) * 2018-09-14 2020-03-24 哈尔滨工业大学(深圳) Attribute generation countermeasure network and matching clothing generation method based on same
CN109388565B (en) * 2018-09-27 2021-08-06 西安电子科技大学 Software system performance optimization method based on generating type countermeasure network
CN109388565A (en) * 2018-09-27 2019-02-26 西安电子科技大学 Software system performance optimization method based on production confrontation network
CN109147810A (en) * 2018-09-30 2019-01-04 百度在线网络技术(北京)有限公司 Establish the method, apparatus, equipment and computer storage medium of speech enhan-cement network
WO2020073758A1 (en) * 2018-10-10 2020-04-16 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for training machine learning modle, apparatus for video style transfer
CN109583345A (en) * 2018-11-21 2019-04-05 平安科技(深圳)有限公司 Roads recognition method, device, computer installation and computer readable storage medium
CN109583345B (en) * 2018-11-21 2023-09-26 平安科技(深圳)有限公司 Road recognition method, device, computer device and computer readable storage medium
CN109544545A (en) * 2018-11-30 2019-03-29 济南浪潮高新科技投资发展有限公司 A kind of salt mine intelligent detecting method and system based on convolutional neural networks
CN109671125A (en) * 2018-12-17 2019-04-23 电子科技大学 A kind of GAN network model that height merges and the method for realizing text generation image
CN109801228A (en) * 2018-12-18 2019-05-24 合肥阿巴赛信息科技有限公司 A kind of jewelry picture beautification algorithm based on deep learning
CN111340060A (en) * 2018-12-19 2020-06-26 财团法人工业技术研究院 Training method of image generator
CN111340060B (en) * 2018-12-19 2023-03-24 财团法人工业技术研究院 Training method of image generator
CN109685811B (en) * 2018-12-24 2019-12-13 北京大学第三医院 PET/CT high-metabolism lymph node segmentation method based on dual-path U-net convolutional neural network
CN109685811A (en) * 2018-12-24 2019-04-26 北京大学第三医院 PET/CT hypermetabolism lymph node dividing method based on dual path U-net convolutional neural networks
CN109829849B (en) * 2019-01-29 2023-01-31 达闼机器人股份有限公司 Training data generation method and device and terminal
CN109829849A (en) * 2019-01-29 2019-05-31 深圳前海达闼云端智能科技有限公司 A kind of generation method of training data, device and terminal
CN110070587B (en) * 2019-03-05 2023-02-10 浙江工商大学 Pedestrian image generation method based on conditional cascade confrontation generation network
CN110070587A (en) * 2019-03-05 2019-07-30 浙江工商大学 It is a kind of to cascade the pedestrian image generation method that confrontation generates network based on condition
CN110008846B (en) * 2019-03-13 2022-08-30 南京邮电大学 Image processing method
CN110008846A (en) * 2019-03-13 2019-07-12 南京邮电大学 A kind of image processing method
CN111754389A (en) * 2019-03-27 2020-10-09 通用汽车环球科技运作有限责任公司 Semantic preserving style transfer
CN110136063B (en) * 2019-05-13 2023-06-23 南京信息工程大学 Single image super-resolution reconstruction method based on condition generation countermeasure network
CN110136063A (en) * 2019-05-13 2019-08-16 南京信息工程大学 A kind of single image super resolution ratio reconstruction method generating confrontation network based on condition
CN110211203A (en) * 2019-06-10 2019-09-06 大连民族大学 The method of the Chinese character style of confrontation network is generated based on condition
CN110298844A (en) * 2019-06-17 2019-10-01 艾瑞迈迪科技石家庄有限公司 X-ray contrastographic picture blood vessel segmentation and recognition methods and device
CN110298844B (en) * 2019-06-17 2021-06-29 艾瑞迈迪科技石家庄有限公司 X-ray radiography image blood vessel segmentation and identification method and device
CN110347819A (en) * 2019-06-21 2019-10-18 同济大学 A kind of text snippet generation method based on positive negative sample dual training
CN110533580A (en) * 2019-08-08 2019-12-03 西安交通大学 A kind of image Style Transfer method generating neural network based on confrontation
CN110634108A (en) * 2019-08-30 2019-12-31 北京工业大学 Composite degraded live webcast video enhancement method based on element-cycle consistency countermeasure network
CN110634108B (en) * 2019-08-30 2023-01-20 北京工业大学 Composite degraded network live broadcast video enhancement method based on element-cycle consistency confrontation network
CN110826059A (en) * 2019-09-19 2020-02-21 浙江工业大学 Method and device for defending black box attack facing malicious software image format detection model
CN110826059B (en) * 2019-09-19 2021-10-15 浙江工业大学 Method and device for defending black box attack facing malicious software image format detection model
CN110619315A (en) * 2019-09-24 2019-12-27 重庆紫光华山智安科技有限公司 Training method and device of face recognition model and electronic equipment
CN111028306A (en) * 2019-11-06 2020-04-17 杭州电子科技大学 AR2U-Net neural network-based rapid magnetic resonance imaging method
CN111028306B (en) * 2019-11-06 2023-07-14 杭州电子科技大学 AR2U-Net neural network-based rapid magnetic resonance imaging method
CN111079562A (en) * 2019-11-27 2020-04-28 浙江大学 Multi-stage data generation self-circulation financial invoice text intelligent identification system and method
CN111047507A (en) * 2019-11-29 2020-04-21 北京达佳互联信息技术有限公司 Training method of image generation model, image generation method and device
CN111260620A (en) * 2020-01-14 2020-06-09 三峡大学 Image anomaly detection method and device and electronic equipment
CN111260620B (en) * 2020-01-14 2023-08-18 三峡大学 Image anomaly detection method and device and electronic equipment
CN111242953A (en) * 2020-01-17 2020-06-05 陕西师范大学 MR image segmentation method and device based on condition generation countermeasure network
CN111260545A (en) * 2020-01-20 2020-06-09 北京百度网讯科技有限公司 Method and device for generating image
CN111275784A (en) * 2020-01-20 2020-06-12 北京百度网讯科技有限公司 Method and device for generating image
CN111402121A (en) * 2020-03-20 2020-07-10 上海眼控科技股份有限公司 Image style conversion method and device, computer equipment and storage medium
CN111784592A (en) * 2020-05-19 2020-10-16 知昇(上海)人工智能科技有限公司 Automatic design image generation method based on GAN
CN111724299B (en) * 2020-05-21 2023-08-08 同济大学 Deep learning-based super-reality sense painting image style migration method
CN111724299A (en) * 2020-05-21 2020-09-29 同济大学 Super-realistic painting image style migration method based on deep learning
CN111583168A (en) * 2020-06-18 2020-08-25 上海眼控科技股份有限公司 Image synthesis method, image synthesis device, computer equipment and storage medium
CN111784565B (en) * 2020-07-01 2021-10-29 北京字节跳动网络技术有限公司 Image processing method, migration model training method, device, medium and equipment
CN111784565A (en) * 2020-07-01 2020-10-16 北京字节跳动网络技术有限公司 Image processing method, migration model training method, device, medium and equipment
CN111899251A (en) * 2020-08-06 2020-11-06 中国科学院深圳先进技术研究院 Copy-move type forged image detection method for distinguishing forged source and target area
CN112001122B (en) * 2020-08-26 2023-09-26 合肥工业大学 Non-contact physiological signal measurement method based on end-to-end generation countermeasure network
CN112001122A (en) * 2020-08-26 2020-11-27 合肥工业大学 Non-contact physiological signal measuring method based on end-to-end generation countermeasure network
CN112288622A (en) * 2020-10-29 2021-01-29 中山大学 Multi-scale generation countermeasure network-based camouflaged image generation method
CN112731522A (en) * 2020-12-14 2021-04-30 中国地质大学(武汉) Intelligent recognition method, device and equipment for seismic stratum and storage medium
CN113538216A (en) * 2021-06-16 2021-10-22 电子科技大学 Image style migration method based on attribute decomposition
CN113538224A (en) * 2021-09-14 2021-10-22 深圳市安软科技股份有限公司 Image style migration method and device based on generation countermeasure network and related equipment
CN113538224B (en) * 2021-09-14 2022-01-14 深圳市安软科技股份有限公司 Image style migration method and device based on generation countermeasure network and related equipment
CN113724132A (en) * 2021-11-03 2021-11-30 浙江宇视科技有限公司 Image style migration processing method and device, electronic equipment and storage medium
CN114863225B (en) * 2022-07-06 2022-10-04 腾讯科技(深圳)有限公司 Image processing model training method, image processing model generation device, image processing model equipment and image processing model medium
CN114863225A (en) * 2022-07-06 2022-08-05 腾讯科技(深圳)有限公司 Image processing model training method, image processing model generation device, image processing equipment and image processing medium
CN115546589A (en) * 2022-11-29 2022-12-30 浙江大学 Image generation method based on graph neural network
CN115588070A (en) * 2022-12-12 2023-01-10 南方科技大学 Three-dimensional image stylized migration method and terminal
CN115588070B (en) * 2022-12-12 2023-03-14 南方科技大学 Three-dimensional image stylized migration method and terminal
CN116630140A (en) * 2023-03-31 2023-08-22 南京信息工程大学 Method, equipment and medium for realizing animation portrait humanization based on condition generation countermeasure network

Similar Documents

Publication Publication Date Title
AU2017101166A4 (en) A Method For Real-Time Image Style Transfer Based On Conditional Generative Adversarial Networks
Dolhansky et al. Eye in-painting with exemplar generative adversarial networks
Anantrasirichai et al. Artificial intelligence in the creative industries: a review
Islam et al. Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception
Vitoria et al. Chromagan: Adversarial picture colorization with semantic class distribution
Sangkloy et al. Scribbler: Controlling deep image synthesis with sketch and color
US10839581B2 (en) Computer-implemented method for generating composite image, apparatus for generating composite image, and computer-program product
Li et al. A closed-form solution to photorealistic image stylization
Wang et al. Multimodal transfer: A hierarchical deep convolutional neural network for fast artistic style transfer
Korshunova et al. Fast face-swap using convolutional neural networks
Liu et al. Structure-guided arbitrary style transfer for artistic image and video
Thasarathan et al. Automatic temporally coherent video colorization
Rodriguez-Pardo et al. Seamlessgan: Self-supervised synthesis of tileable texture maps
Liu et al. Image neural style transfer with preserving the salient regions
Li et al. High-resolution network for photorealistic style transfer
CN113129240B (en) Motion blur removing method for industrial packaging characters
Mun et al. Texture preserving photo style transfer network
Kim et al. Game effect sprite generation with minimal data via conditional GAN
Tous Pictonaut: movie cartoonization using 3D human pose estimation and GANs
US20230319223A1 (en) Method and system for deep learning based face swapping with multiple encoders
Althbaity et al. Colorization Of Grayscale Images Using Deep Learning
Zhou et al. A Semi‐Procedural Convolutional Material Prior
Dey Image Processing Masterclass with Python: 50+ Solutions and Techniques Solving Complex Digital Image Processing Challenges Using Numpy, Scipy, Pytorch and Keras (English Edition)
GAO et al. Bidirectional Mapping Augmentation Algorithm for Synthetic Images Based on Generative Adversarial Network
US20240029354A1 (en) Facial texture synthesis for three-dimensional morphable models

Legal Events

Date Code Title Description
FGI Letters patent sealed or granted (innovation patent)
MK22 Patent ceased section 143a(d), or expired - non payment of renewal fee or expiry