CN109840926B - Image generation method, device and equipment - Google Patents
Image generation method, device and equipment Download PDFInfo
- Publication number
- CN109840926B CN109840926B CN201811646932.8A CN201811646932A CN109840926B CN 109840926 B CN109840926 B CN 109840926B CN 201811646932 A CN201811646932 A CN 201811646932A CN 109840926 B CN109840926 B CN 109840926B
- Authority
- CN
- China
- Prior art keywords
- image
- sample image
- sample
- gan model
- generated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an image generation method, device and equipment, wherein the method comprises the following steps: acquiring a first image; generating a second image through a pre-trained novel generation type antagonism network GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training. According to the invention, in the process of training a novel GAN model, a generation loop of the GAN network is added, and a novel cost loss function is designed, so that the novel GAN model can solve the problem that the background in the generated image is modified. The novel GAN model is utilized to generate images, so that the generation effect of the images can be enhanced, and the foreground and background mixed phenomena in the generated images are reduced.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image generation method, device and equipment.
Background
GAN (Generative Adversarial Networks, generative antagonism network) is a deep learning model, which is one of the most promising approaches for unsupervised learning on complex distributions in recent years. GAN is currently most commonly used for image generation, such as super-resolution tasks, semantic segmentation, and so forth. When the GAN is used for generating images, one image is input into the GAN, and the GAN learns and outputs a new image in a game through a generator and a discriminator in the framework.
However, in the current related art, the image generated by using GAN has a phenomenon that the foreground and the background are mixed, and the effect of generating the image is to be improved.
Disclosure of Invention
In order to solve the problems, the invention provides an image generation method, an image generation device and image generation equipment, wherein in the process of training a novel GAN model, a generation loop of a GAN network is added, a novel cost loss function is designed, the generation effect of generating an image by GAN is enhanced, and the phenomenon of mixing the foreground and the background in the generated image is reduced. The present invention solves the above problems by:
in a first aspect, an embodiment of the present invention provides an image generating method, including:
acquiring a first image;
generating a second image through a pre-trained generation type antagonism network novel GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where before the generating, according to the first image, a second image by using a pre-trained new GAN model, the method further includes:
obtaining a training sample set, the training sample set comprising a plurality of image pairs;
and training a novel GAN model through the training sample set, wherein the novel GAN model comprises a first generator, a second generator, a first discriminator and a second discriminator.
With reference to the first possible implementation manner of the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the training the new GAN model through the training sample set includes:
randomly acquiring a first sample image and a second sample image included in a group of image pairs from the training sample set;
training the novel GAN model according to the first sample image and the second sample image;
calculating a cost value of a novel cost loss function corresponding to the novel GAN model;
and judging whether to terminate training of the novel GAN model according to the cost value.
With reference to the second possible implementation manner of the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where the training the new GAN model according to the first sample image and the second sample image includes:
generating a first generated image through a first generator included in the novel GAN model according to the first sample image, and judging the first generated image through a first judging device included in the novel GAN model;
generating a second generated image through a second generator included in the novel GAN model according to the first generated image;
judging the first sample image through a second judging device included in the novel GAN model;
generating a third generated image by the second generator from the first sample image;
generating a fourth generated image by the second generator according to the second sample image, and judging the fourth generated image by the second judging device;
generating a fifth generated image by the first generator according to the fourth generated image;
discriminating the second sample image by the first discriminator;
and generating a sixth generated image through the first generator according to the second sample image.
With reference to the third possible implementation manner of the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the calculating a cost value of a new cost loss function corresponding to the new GAN model includes:
calculating a loss value corresponding to the first sample image according to the first sample image and the third generated image;
calculating a loss value corresponding to the second sample image according to the second sample image and the sixth generated image;
and calculating the cost value of the novel cost loss function corresponding to the novel GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image.
With reference to the fourth possible implementation manner of the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where the calculating, according to the first sample image and the third generated image, a loss value corresponding to the first sample image includes:
calculating a difference value between the first sample image and the third generated image to obtain a difference matrix corresponding to the first sample image;
calculating an average value of all pixel values included in the difference matrix;
and determining the average value as a loss value corresponding to the first sample image.
With reference to the fourth possible implementation manner of the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where the calculating, according to a loss value corresponding to the first sample image and a loss value corresponding to the second sample image, a cost value of a new cost loss function corresponding to the new GAN model includes:
calculating a cost value of a novel cost loss function corresponding to the novel GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image through a formula (1);
L＝L(A,A3)+L(B,B3)+LOSS…(1)
in the formula (1), L is the cost value, L (a, A3) is the LOSS value corresponding to the first sample image, L (B, B3) is the LOSS value corresponding to the second sample image, and LOSS is the original cost function value.
In a second aspect, an embodiment of the present invention provides an image generating apparatus, including:
the acquisition module is used for acquiring a first image;
and the generation module is used for generating a second image through a pre-trained generation type antagonism network novel GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training.
With reference to the second aspect, an embodiment of the present invention provides a first possible implementation manner of the second aspect, where the apparatus further includes:
the model training module is used for acquiring a training sample set, wherein the training sample set comprises a plurality of groups of image pairs; and training a novel GAN model through the training sample set, wherein the novel GAN model comprises a first generator, a second generator, a first discriminator and a second discriminator.
In a third aspect, an embodiment of the present invention provides an image generating apparatus, including a memory and a processor;
the memory has executable program code stored therein;
the processor reads the executable program code and runs a program corresponding to the executable program code to implement the image generating method according to the first aspect or any possible implementation manner of the first aspect.
In the embodiment of the invention, a first image is acquired; generating a second image through a pre-trained novel generation type antagonism network GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training. According to the invention, in the process of training a novel GAN model, a generation loop of the GAN network is added, and a novel cost loss function is designed, so that the novel GAN model can solve the problem that the background in the generated image is modified. The novel GAN model is utilized to generate images, so that the generation effect of the images can be enhanced, and the foreground and background mixed phenomena in the generated images are reduced.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 shows a schematic structural diagram of a novel GAN model according to embodiment 1 of the present invention;
fig. 2 shows a schematic diagram of a training flow of the novel GAN model provided in embodiment 1 of the present invention;
fig. 3 is a flowchart of an image generating method according to embodiment 1 of the present invention;
fig. 4 is a schematic diagram showing the structure of an image generating apparatus according to embodiment 2 of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Example 1
The embodiment of the invention provides an image generation method, which relates to a novel GAN network structure, increases a generation loop of the novel GAN network, designs a novel cost loss function, can enhance the novel GAN generation effect, and reduces the phenomenon of foreground and background mixing in the generated image.
Before generating an image by the method, the novel GAN model needs to be trained firstly, specifically by the following operations of steps A1 and A2, including:
a1: a training sample set is acquired, the training sample set comprising a plurality of image pairs.
A large number of image pairs are acquired, and two images are included in one image pair, and the two images included in the one image pair are called a first sample image and a second sample image according to the embodiment of the present invention. The objects depicted by the foreground in the first sample image and the second sample image are objects with the same theme, and the foreground in the first sample image and the second sample image is a horse or a cat.
A2: a novel GAN model is trained by training a sample set, the novel GAN model including a first generator, a second generator, a first discriminant, and a second discriminant.
As shown in fig. 1, the novel GAN model in the embodiment of the invention includes a first generator, a second generator, a first arbiter and a second arbiter.
Specifically, a first sample image and a second sample image included in a group of image pairs are randomly acquired from a training sample set; training a novel GAN model according to the first sample image and the second sample image; calculating a cost value of a novel cost loss function corresponding to the novel GAN model; and judging whether to terminate training of the novel GAN model according to the cost value. And if the cost value is smaller than the preset threshold value, stopping training the novel GAN model to obtain a final trained novel GAN model. If the cost value is greater than or equal to a preset threshold value, a group of images including a first sample image and a second sample image are acquired from the training sample set again, the novel GAN model is trained again by using the newly acquired first sample image and second sample image, and the cost value of a novel cost loss function corresponding to the novel GAN model is calculated, so that training is repeatedly performed on the novel GAN model until the cost value of the novel cost loss function corresponding to the novel GAN model is smaller than the preset threshold value, training on the novel GAN model is terminated, and finally a trained novel GAN model is obtained.
As shown in fig. 2, for each acquired first sample image and second sample image, the novel GAN model is trained by the following operations of steps B1-B8, specifically including:
b1: according to the first sample image, a first generated image is generated through a first generator included in the novel GAN model, and the first generated image is judged through a first judging device included in the novel GAN model.
The first sample image is input into a first generator included in the novel GAN model, and the first generator generates a first generated image from the first sample image. Wherein the first generated image is an object of the same subject as the object depicted by the foreground of the first sample image, and the first generated image is an image that approximates the second sample image. Inputting the first generated image into a first discriminator included in the novel GAN model, and discriminating the first generated image by the first discriminator to determine whether the first generated image is a real image or not.
The first generation image which approximates to the second sample image is generated through a first generator according to the first sample image, and the first generation image is judged through a first judging device. That is, the training first generator generates a new image from an image that is not a true image, and trains the first arbiter with the false image.
B2: and generating a second generated image through a second generator included in the novel GAN model according to the first generated image.
And inputting the generated first generated image into a second generator included in the novel GAN model, wherein the second generator generates a second generated image according to the first generated image, and the second generated image and an object drawn by the foreground of the first generated image are objects with the same theme. Wherein the second generated image is an image similar to the first sample image.
B3: the first sample image is discriminated by a second discriminator included in the novel GAN model.
The first sample image is input to a second discriminator, which discriminates whether the first sample image is a true image or not. I.e. training the second discriminant with the actual image.
B4: a third generated image is generated by a second generator from the first sample image.
The first sample image is input into a second generator, which generates a third generated image from the first sample image. Wherein the third generated image is an image similar to the first sample image. I.e. a third generated image similar to the first sample data itself is generated by the second generator.
In the embodiment of the invention, after the third generated image is generated, the difference value between the first sample image and the third generated image is also calculated, so as to obtain a difference matrix corresponding to the first sample image. Namely, calculating the difference value between the pixel matrix corresponding to the first sample image and the pixel matrix corresponding to the third generated image to obtain a difference value matrix corresponding to the first sample image. Calculating an average value of all pixel values included in the difference matrix; the average value is determined as a loss value corresponding to the first sample image.
Specifically, the loss value corresponding to the first sample image is calculated by the following formula (2):
L(A,A3)＝E[A-A3]…(2)
in the above formula (2), A is a pixel matrix corresponding to the first sample image, A3 is a pixel matrix corresponding to the third generated image, E [ a-A3] is a difference matrix corresponding to the first sample image, and L (A, A3) is a loss value corresponding to the first sample image.
The loss value corresponding to the first sample image can represent the degree of mixing of the foreground and the background of the image generated by the second generator.
B5: a fourth generated image is generated by a second generator from the second sample image, and the fourth generated image is discriminated by a second discriminator.
The second sample image is input into a second generator, which generates a fourth generated image from the second sample image. Wherein the fourth generated image is an object of the same subject as the object depicted by the foreground of the second sample image, and the fourth generated image is an image similar to the first sample image. And inputting the fourth generated image into a second discriminator included in the novel GAN model, and discriminating the fourth generated image by the second discriminator to determine whether the fourth generated image is a real image or not.
The step is to generate a fourth generated image similar to the first sample image by a second generator according to the second sample image, and judge the fourth generated image by a second discriminator. That is, the training second generator generates a new image from an image that is not a true image, and trains the second arbiter with the false image.
B6: and generating a fifth generated image by the first generator according to the fourth generated image.
And inputting the generated fourth generated image into a first generator included in the novel GAN model, wherein the first generator generates a fifth generated image according to the fourth generated image, and objects drawn by the foreground of the fifth generated image and the fourth generated image are objects with the same theme. Wherein the fifth generated image is an image similar to the second sample image.
B7: the second sample image is discriminated by the first discriminator.
The second sample image is input to a first discriminator, which discriminates whether the second sample image is a true image. I.e. training the first discriminant with the actual image.
B8: a sixth generated image is generated by the first generator from the second sample image.
The second sample image is input into the first generator, which generates a sixth generated image from the second sample image. Wherein the sixth generated image is an image similar to the second sample image. I.e. a sixth generated image similar to the second sample data itself is generated by the first generator.
In the embodiment of the invention, after the sixth generated image is generated, the difference value between the second sample image and the sixth generated image is also calculated, so as to obtain a difference matrix corresponding to the second sample image. Namely, calculating the difference value between the pixel matrix corresponding to the second sample image and the pixel matrix corresponding to the sixth generated image to obtain a difference value matrix corresponding to the second sample image. Calculating an average value of all pixel values included in the difference matrix; the average value is determined as a loss value corresponding to the second sample image.
Specifically, the loss value corresponding to the first sample image is calculated by the following formula (3):
L(B,B3)＝E[B-B3]…(3)
in the above formula (3), B is a pixel matrix corresponding to the second sample image, B3 is a pixel matrix corresponding to the sixth generated image, E [ B-B3] is a difference matrix corresponding to the second sample image, and L (B, B3) is a loss value corresponding to the second sample image.
The corresponding loss value of the second sample image can reflect the mixing degree of the foreground and the background of the image generated by the first generator.
Training the novel GAN model by using the first sample image and the second sample image, and calculating a loss value corresponding to the first sample image and a loss value corresponding to the second sample image, and then calculating a cost value of a novel cost loss function corresponding to the novel GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image.
Specifically, calculating a cost value of a novel cost loss function corresponding to the novel GAN model by the following formula (1);
L＝L(A,A3)+L(B,B3)+LOSS…(1)
in the formula (1), L is a cost value, L (a, A3) is a LOSS value corresponding to the first sample image, L (B, B3) is a LOSS value corresponding to the second sample image, and LOSS is an original cost function value. The original cost function value LOSS may be a cost function value of the cycle novel GAN network.
And (3) repeatedly training the novel GAN model by utilizing a plurality of groups of data pairs included in the training data according to the operations of the steps (B1-B5) until the cost value of the novel cost loss function corresponding to the novel GAN model is smaller than a preset threshold value, stopping the training process, and obtaining the trained novel GAN model. Thereafter, as shown in fig. 3, an image is generated by using the pre-trained novel GAN model through the following operations of steps 101 and 102, which specifically include:
step 101: a first image is acquired.
Step 102: generating a second image according to the first image through a pre-trained novel GAN model, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training.
The first image is input into a first generator and/or a second generator included in the novel GAN model to generate a new image, and the first discriminator and/or the second discriminator are used for discriminating the new image. Because the novel pre-trained GAN model is formed by controlling and training the novel cost loss function, the cost loss in the image generation process is small, and the phenomenon of foreground and background mixing in the generated image is greatly reduced.
In the embodiment of the invention, a first image is acquired; generating a second image through a pre-trained novel generation type antagonism network GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training. According to the invention, in the process of training a novel GAN model, a generation loop of the GAN network is added, and a novel cost loss function is designed, so that the novel GAN model can solve the problem that the background in the generated image is modified. The novel GAN model is utilized to generate images, so that the generation effect of the images can be enhanced, and the foreground and background mixed phenomena in the generated images are reduced.
Example 2
Referring to fig. 4, an embodiment of the present application provides an image generating apparatus for performing the image generating method provided in embodiment 1, the apparatus including:
an acquisition module 20 for acquiring a first image;
the generating module 21 is configured to generate, from the first image, a second image through a pre-trained generation type counternetwork new GAN model, where the pre-trained new GAN model is obtained through new cost loss function control training.
In an embodiment of the present invention, the apparatus further includes:
the model training module is used for acquiring a training sample set, and the training sample set comprises a plurality of image pairs; the novel GAN model is trained by training the sample set, the novel GAN model including a first generator, a second generator, a first discriminant, and a second discriminant.
The model training module comprises:
an acquisition unit for randomly acquiring a first sample image and a second sample image included in a set of image pairs from a training sample set;
the training unit is used for training a novel GAN model according to the first sample image and the second sample image;
the calculating unit is used for calculating a cost value of a novel cost loss function corresponding to the novel GAN model;
and the judging unit is used for judging whether to terminate training of the novel GAN model according to the cost value.
The training unit is configured to generate a first generated image according to the first sample image through a first generator included in the novel GAN model, and determine the first generated image through a first discriminator included in the novel GAN model; generating a second generated image through a second generator included in the novel GAN model according to the first generated image; judging the first sample image by a second judging device included in the novel GAN model; generating a third generated image from the first sample image by a second generator; generating a fourth generated image by a second generator according to the second sample image, and judging the fourth generated image by a second judging device; generating a fifth generated image by the first generator according to the fourth generated image; judging a second sample image by a first judging device; a sixth generated image is generated by the first generator from the second sample image.
The calculating unit is used for calculating a loss value corresponding to the first sample image according to the first sample image and the third generated image; calculating a loss value corresponding to the second sample image according to the second sample image and the sixth generated image; and calculating the cost value of the novel cost loss function corresponding to the novel GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image.
The calculating unit is further configured to calculate a difference between the first sample image and the third generated image, to obtain a difference matrix corresponding to the first sample image; calculating an average value of all pixel values included in the difference matrix; and determining the average value as a loss value corresponding to the first sample image.
The calculating unit is further configured to calculate, according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image, a cost value of a new cost loss function corresponding to the new GAN model according to formula (1);
L＝L(A,A3)+L(B,B3)+LOSS…(1)
in the formula (1), L is a cost value, L (a, A3) is a LOSS value corresponding to the first sample image, L (B, B3) is a LOSS value corresponding to the second sample image, and LOSS is an original cost function value.
In the embodiment of the invention, a first image is acquired; generating a second image through a pre-trained novel generation type antagonism network GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training. According to the invention, in the process of training a novel GAN model, a generation loop of the GAN network is added, and a novel cost loss function is designed, so that the novel GAN model can solve the problem that the background in the generated image is modified. The novel GAN model is utilized to generate images, so that the generation effect of the images can be enhanced, and the foreground and background mixed phenomena in the generated images are reduced.
Example 3
An embodiment of the present invention provides an image generating apparatus, which includes one or more processors, and one or more storage devices, where one or more programs are stored in the one or more storage devices, and when the one or more programs are loaded and executed by the one or more processors, the image generating method provided in the foregoing embodiment 1 is implemented.
In the embodiment of the invention, a first image is acquired; generating a second image through a pre-trained novel generation type antagonism network GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training. According to the invention, in the process of training a novel GAN model, a generation loop of the GAN network is added, and a novel cost loss function is designed, so that the novel GAN model can solve the problem that the background in the generated image is modified. The novel GAN model is utilized to generate images, so that the generation effect of the images can be enhanced, and the foreground and background mixed phenomena in the generated images are reduced.
Example 4
An embodiment of the present invention provides a computer-readable storage medium having stored therein an executable program which, when loaded and executed by a processor, realizes the image generation provided in the above embodiment 1.
In the embodiment of the invention, a first image is acquired; generating a second image through a pre-trained novel generation type antagonism network GAN model according to the first image, wherein the pre-trained novel GAN model is obtained through novel cost loss function control training. According to the invention, in the process of training a novel GAN model, a generation loop of the GAN network is added, and a novel cost loss function is designed, so that the novel GAN model can solve the problem that the background in the generated image is modified. The novel GAN model is utilized to generate images, so that the generation effect of the images can be enhanced, and the foreground and background mixed phenomena in the generated images are reduced.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may also be used with the teachings herein. The required structure for the construction of such devices is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in the creation means of a virtual machine according to an embodiment of the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (5)
1. An image generation method, the method comprising:
acquiring a first image;
generating a second image through a pre-trained generation type antagonism network GAN model according to the first image, wherein the pre-trained GAN model is obtained through cost loss function control training;
wherein before the second image is generated according to the first image and through the pre-trained GAN model, the method further comprises:
obtaining a training sample set, the training sample set comprising a plurality of image pairs;
randomly acquiring a first sample image and a second sample image included in a group of image pairs from the training sample set;
training the GAN model according to the first sample image and the second sample image, wherein the GAN model comprises a first generator, a second generator, a first discriminator and a second discriminator;
wherein said training said GAN model from said first sample image and said second sample image comprises:
generating a first generated image through a first generator included in the GAN model according to the first sample image, and judging the first generated image through a first judging device included in the GAN model;
generating a second generated image through a second generator included in the GAN model according to the first generated image;
judging the first sample image through a second judging device included in the GAN model;
generating a third generated image by the second generator from the first sample image;
generating a fourth generated image by the second generator according to the second sample image, and judging the fourth generated image by the second judging device;
generating a fifth generated image by the first generator according to the fourth generated image;
discriminating the second sample image by the first discriminator;
generating a sixth generated image from the second sample image by the first generator;
calculating a loss value corresponding to the first sample image according to the first sample image and the third generated image;
calculating a loss value corresponding to the second sample image according to the second sample image and the sixth generated image;
calculating a cost value of a cost loss function corresponding to the GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image;
and judging whether to terminate training of the GAN model according to the cost value.
2. The method of claim 1, wherein calculating a loss value corresponding to the first sample image from the first sample image and the third generated image comprises:
calculating a difference value between the first sample image and the third generated image to obtain a difference matrix corresponding to the first sample image;
calculating an average value of all pixel values included in the difference matrix;
and determining the average value as a loss value corresponding to the first sample image.
3. The method according to claim 1, wherein the calculating the cost value of the cost-loss function corresponding to the GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image includes:
according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image, calculating a cost value of a cost loss function corresponding to the GAN model through a formula (1);
L＝L(A,A3)+L(B,B3)+LOSS (1)
in the formula (1), L is the cost value, L (a, A3) is the LOSS value corresponding to the first sample image, L (B, B3) is the LOSS value corresponding to the second sample image, LOSS is the original cost function value, a is the pixel matrix corresponding to the first sample image, A3 is the pixel matrix corresponding to the third generated image, B is the pixel matrix corresponding to the second sample image, and B3 is the pixel matrix corresponding to the sixth generated image.
4. An image generation apparatus, the apparatus comprising:
the acquisition module is used for acquiring a first image;
the generation module is used for generating a second image through a pre-trained generation type antagonism network GAN model according to the first image, wherein the pre-trained GAN model is obtained through cost loss function control training;
wherein the apparatus further comprises:
the model training module is used for acquiring a training sample set, wherein the training sample set comprises a plurality of groups of image pairs; training a GAN model by the training sample set, the GAN model comprising a first generator, a second generator, a first arbiter, and a second arbiter;
the model training module comprises:
an acquisition unit for randomly acquiring a first sample image and a second sample image included in a set of image pairs from a training sample set;
the training unit is used for training the GAN model according to the first sample image and the second sample image;
the training unit is specifically configured to generate a first generated image according to a first sample image through a first generator included in a GAN model, and judge the first generated image through a first discriminator included in the GAN model; generating a second generated image through a second generator included in the GAN model according to the first generated image; judging the first sample image by a second judging device included in the GAN model; generating a third generated image from the first sample image by a second generator; generating a fourth generated image by a second generator according to the second sample image, and judging the fourth generated image by a second judging device; generating a fifth generated image by the first generator according to the fourth generated image; judging a second sample image by a first judging device; generating a sixth generated image from the second sample image by the first generator;
a calculating unit, configured to calculate a loss value corresponding to the first sample image according to the first sample image and the third generated image; calculating a loss value corresponding to the second sample image according to the second sample image and the sixth generated image; and calculating the cost value of the cost loss function corresponding to the GAN model according to the loss value corresponding to the first sample image and the loss value corresponding to the second sample image.
5. An image generating apparatus comprising a memory and a processor;
the memory has executable program code stored therein;
the processor reads the executable program code, runs a program corresponding to the executable program code, to implement the image generation method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811646932.8A CN109840926B (en) | 2018-12-29 | 2018-12-29 | Image generation method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811646932.8A CN109840926B (en) | 2018-12-29 | 2018-12-29 | Image generation method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109840926A CN109840926A (en) | 2019-06-04 |
| CN109840926B true CN109840926B (en) | 2023-06-20 |
Family
ID=66883593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811646932.8A Active CN109840926B (en) | 2018-12-29 | 2018-12-29 | Image generation method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109840926B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114429664A (en) * | 2022-01-29 | 2022-05-03 | 脸萌有限公司 | Video generation method and training method of video generation model |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107123151A (en) * | 2017-04-28 | 2017-09-01 | 深圳市唯特视科技有限公司 | A kind of image method for transformation based on variation autocoder and generation confrontation network |
| CN108182657A (en) * | 2018-01-26 | 2018-06-19 | 深圳市唯特视科技有限公司 | A kind of face-image conversion method that confrontation network is generated based on cycle |
| CN108288072A (en) * | 2018-01-26 | 2018-07-17 | 深圳市唯特视科技有限公司 | A kind of facial expression synthetic method based on generation confrontation network |
| CN108491809A (en) * | 2018-03-28 | 2018-09-04 | 百度在线网络技术（北京）有限公司 | The method and apparatus for generating model for generating near-infrared image |
| CN108665414A (en) * | 2018-05-10 | 2018-10-16 | 上海交通大学 | Natural scene picture generation method |
| CN108875511A (en) * | 2017-12-01 | 2018-11-23 | 北京迈格威科技有限公司 | Method, apparatus, system and the computer storage medium that image generates |
| CN108875766A (en) * | 2017-11-29 | 2018-11-23 | 北京旷视科技有限公司 | Method, apparatus, system and the computer storage medium of image procossing |
| CN114120028A (en) * | 2021-10-26 | 2022-03-01 | 中国电子科技集团公司第五十四研究所 | Countermeasure sample generation method based on double-layer generation countermeasure network |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2018063504A (en) * | 2016-10-12 | 2018-04-19 | 株式会社リコー | Generation model learning method, device and program |
| US20180247201A1 (en) * | 2017-02-28 | 2018-08-30 | Nvidia Corporation | Systems and methods for image-to-image translation using variational autoencoders |
| US10474929B2 (en) * | 2017-04-25 | 2019-11-12 | Nec Corporation | Cyclic generative adversarial network for unsupervised cross-domain image generation |
| KR102403494B1 (en) * | 2017-04-27 | 2022-05-27 | 에스케이텔레콤 주식회사 | Method for learning Cross-domain Relations based on Generative Adversarial Network |
-
2018
- 2018-12-29 CN CN201811646932.8A patent/CN109840926B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107123151A (en) * | 2017-04-28 | 2017-09-01 | 深圳市唯特视科技有限公司 | A kind of image method for transformation based on variation autocoder and generation confrontation network |
| CN108875766A (en) * | 2017-11-29 | 2018-11-23 | 北京旷视科技有限公司 | Method, apparatus, system and the computer storage medium of image procossing |
| CN108875511A (en) * | 2017-12-01 | 2018-11-23 | 北京迈格威科技有限公司 | Method, apparatus, system and the computer storage medium that image generates |
| CN108182657A (en) * | 2018-01-26 | 2018-06-19 | 深圳市唯特视科技有限公司 | A kind of face-image conversion method that confrontation network is generated based on cycle |
| CN108288072A (en) * | 2018-01-26 | 2018-07-17 | 深圳市唯特视科技有限公司 | A kind of facial expression synthetic method based on generation confrontation network |
| CN108491809A (en) * | 2018-03-28 | 2018-09-04 | 百度在线网络技术（北京）有限公司 | The method and apparatus for generating model for generating near-infrared image |
| CN108665414A (en) * | 2018-05-10 | 2018-10-16 | 上海交通大学 | Natural scene picture generation method |
| CN114120028A (en) * | 2021-10-26 | 2022-03-01 | 中国电子科技集团公司第五十四研究所 | Countermeasure sample generation method based on double-layer generation countermeasure network |
Non-Patent Citations (7)
| Title |
|---|
| Attribute-Guided Face Generation Using Conditional CycleGAN;Yongyi Lu et al.;《arXiv:1705.09966》;20170528;1-16 * |
| Generative Adversarial Networks;Mathew Salvaris et al.;《Deep Learning with Azure》;20180825;187-208 * |
| Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping;Huan FU et al.;《2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)》;20200109 * |
| Unsupervised Transformation Network Based on GANs for Target-Domain Oriented Image Translation;HONGWEI GE ET AL.;《IEEE ACCESS》;20181109;61342-61350 * |
| 基于生成对抗网络的音频音质提升方法;张逸 等;《计算机工程与应用》;20181201;240-244+270 * |
| 基于贝叶斯生成对抗网络的背景消减算法;郑文博 等;《自动化学报》;20180531(第5期);878-890 * |
| 小样本条件下基于深度森林学习模型的典型军事目标识别方法;陈龙 等;《中国电子科学研究院学报》;20190331(第3期);240-244+270 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109840926A (en) | 2019-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110287479A (en) | Name entity recognition method, electronic device and storage medium | |
| CN108021806B (en) | Malicious installation package identification method and device | |
| CN110096617B (en) | Video classification method and device, electronic equipment and computer-readable storage medium | |
| CN110246198B (en) | Method and device for generating character selection verification code, electronic equipment and storage medium | |
| CN112241667A (en) | Image detection method, device, equipment and storage medium | |
| CN109840926B (en) | Image generation method, device and equipment | |
| CN111881706B (en) | Living body detection, image classification and model training method, device, equipment and medium | |
| CN113987429A (en) | Copyright verification method of neural network model based on watermark embedding | |
| CN112398674B (en) | Method and device for generating VNFD configuration template for describing virtual network functions | |
| CN110852351A (en) | Image-based garbage classification method and device, terminal equipment and storage medium | |
| CN112528108A (en) | Model training system, gradient aggregation method and device in model training | |
| CN113516239A (en) | Model training method and device, storage medium and electronic equipment | |
| CN113222043A (en) | Image classification method, device, equipment and storage medium | |
| CN112966754A (en) | Sample screening method, sample screening device and terminal equipment | |
| CN112416301A (en) | Deep learning model development method and device and computer readable storage medium | |
| CN114972375B (en) | Training method and device for image generation model, equipment and storage medium | |
| CN115713669A (en) | Image classification method and device based on inter-class relation, storage medium and terminal | |
| CN115758245A (en) | Multi-mode data classification method, device, equipment and storage medium | |
| CN111143148B (en) | Model parameter determining method, device and storage medium | |
| CN110069770B (en) | Data processing system, method and computer equipment | |
| CN113240032A (en) | Image classification method, device, equipment and storage medium | |
| CN113762382B (en) | Model training and scene recognition method, device, equipment and medium | |
| CN113591917B (en) | Data enhancement method and device | |
| CN116363262B (en) | Image generation method and device and electronic equipment | |
| CN111582446B (en) | System for neural network pruning and neural network pruning processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |