CN114037772A - Training method of image generator, image generation method and device - Google Patents

Training method of image generator, image generation method and device Download PDF

Info

Publication number
CN114037772A
CN114037772A CN202111318799.5A CN202111318799A CN114037772A CN 114037772 A CN114037772 A CN 114037772A CN 202111318799 A CN202111318799 A CN 202111318799A CN 114037772 A CN114037772 A CN 114037772A
Authority
CN
China
Prior art keywords
generator
connection
image
training
discriminator
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111318799.5A
Other languages
Chinese (zh)
Inventor
沈力
刘世伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Information Technology Co Ltd
Original Assignee
Jingdong Technology Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Information Technology Co Ltd filed Critical Jingdong Technology Information Technology Co Ltd
Priority to CN202111318799.5A priority Critical patent/CN114037772A/en
Publication of CN114037772A publication Critical patent/CN114037772A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention discloses a training method of an image generator, an image generation method and an image generation device. The training method of the image generator comprises the following steps: generating an initial generation confrontation network model based on the sparsity of the generator and the discriminator; and performing iterative training on the initially generated confrontation network model based on sample data, updating the connection relation of each network layer in the current iteration generator in the iterative training process, continuing iterative training on the updated generator and the current iteration discriminator until an ending condition is met, and determining the generator after training as a target image generator. By updating the connection with low importance in the training process, the pruning processing of the connection is realized, and the image generator giving consideration to both processing precision and sparsity is obtained after the training is finished. Meanwhile, the sparsity of the generator is kept unchanged, so that the calculated amount in the training process is reduced, and the training efficiency is improved due to the fact that the training parameters are few.

Description

Training method of image generator, image generation method and device
Technical Field
The embodiment of the invention relates to the technical field of deep learning, in particular to a training method of an image generator, an image generation method and an image generation device.
Background
A Generative Adaptive Networks (GAN) is a deep learning model, and is one of the most promising methods for unsupervised learning in complex distribution in recent years. The model passes through (at least) two modules in the framework: mutual game learning of the generator (Generative Model) and discriminator (Discriminative Model) yields reasonably good output.
As the quality of image generation increases, the training cost of generating a countermeasure network also increases. The compressed generated countermeasure network can be obtained by model compression technology of the generated countermeasure network, wherein the model compression technology comprises pruning technology, distillation technology, lottery hypothesis technology and the like.
In the process of implementing the invention, at least the following technical problems are found in the prior art: a dense generative countermeasure network is trained prior to model compression, and the process of model compression includes iteratively performed processes of compression and retraining, resulting in an overall computational effort far exceeding that of training of dense models.
Disclosure of Invention
The embodiment of the invention provides a training method of an image generator, an image generation method and an image generation device, which are used for realizing training to obtain a sparse image generator so as to reduce the storage space and the calculation amount of the image generator.
In a first aspect, an embodiment of the present invention provides a training method for an image generator, including:
generating an initial generation confrontation network model based on the sparsity of the generator and the discriminator;
and performing iterative training on the initially generated countermeasure network model based on sample data, updating the connection relation of each network layer in the generator of the current iteration under the condition of meeting a model updating condition in the iterative training process, continuing iterative training on the updated generator and the discriminator of the current iteration until meeting an ending condition, and determining the generator after training as a target image generator.
In a second aspect, an embodiment of the present invention further provides an image generating method, including:
acquiring an image to be processed;
and inputting the image to be processed into a pre-trained image generator to obtain a target image output by the image generator, wherein the image generator is obtained by training based on the training method of the image generator provided by any embodiment of the invention.
In a third aspect, an embodiment of the present invention further provides a training apparatus for an image generator, including:
the initial model generation module is used for generating an initial generation confrontation network model based on the sparsity of the generator and the discriminator;
and the model training module is used for carrying out iterative training on the initially generated confrontation network model based on sample data, updating the connection relation of each network layer in the generator of the current iteration under the condition of meeting a model updating condition in the iterative training process, continuing iterative training on the updated generator and the discriminator of the current iteration until meeting an ending condition, and determining the generator after training as a target image generator.
In a fourth aspect, an embodiment of the present invention further provides an image generating apparatus, including:
the image to be processed acquisition module is used for acquiring an image to be processed;
and the target image generation module is used for inputting the image to be processed into a pre-trained image generator to obtain a target image output by the image generator, wherein the image generator is obtained by training based on a training method of the image generator provided by any embodiment of the invention.
In a fifth aspect, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the processor implements the training method or the image generation method of the image generator according to any embodiment of the present invention.
In a sixth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the training method or the image generation method of the image generator according to any embodiment of the present invention.
According to the technical scheme provided by the embodiment of the invention, the initially generated confrontation network model is constructed based on the sparsity of the generator and the sparsity of the discriminator, the connection in the initially generated confrontation network model is updated and optimized in the iterative training process of the initially generated confrontation network model, and the updated generated confrontation network model is continuously subjected to iterative training until the end condition is met, so that the target image generator is obtained. By updating the connection with low importance in the training process, the pruning processing of the connection is realized, and the image generator giving consideration to both processing precision and sparsity is obtained after the training is finished. Meanwhile, the sparsity of the generator is kept unchanged, so that the calculated amount in the training process is reduced, and the training efficiency is improved due to the fact that the training parameters are few.
Drawings
Fig. 1 is a schematic flowchart of a training method of an image generator according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an update process of a generator according to an embodiment of the present invention;
FIG. 3 is a flow chart of another training method for an image generator according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of an image generating method according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an training apparatus of an image generator according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an image generating apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Fig. 1 is a flowchart of a training method for an image generator according to an embodiment of the present invention, where the embodiment is applicable to a case of training a generation countermeasure network model to obtain an image generator, and the method may be executed by a training apparatus for an image generator according to an embodiment of the present invention, where the training apparatus for an image generator may be implemented by software and/or hardware, and the training apparatus for an image generator may be configured on an electronic computing device, and specifically includes the following steps:
and S110, generating an initial generation confrontation network model based on the sparsity of the generator and the discriminator.
And S120, performing iterative training on the initially generated confrontation network model based on sample data, and executing the step S130 when a model updating condition is met.
And S130, updating the connection relation of each network layer in the generator of the current iteration, continuing to perform the iterative training of the step S120 on the updated generator and the discriminator of the current iteration until an end condition is met, and determining the generator after training as a target image generator.
The generator and the discriminator respectively comprise a plurality of network layers, each network layer comprises a plurality of neurons, any two neurons connected between the network layers are all provided with a network parameter, the sparsity is used for representing the setting proportion of the network parameter in the model, and the sparsity can be any value between 0% and 100%. The sparsity of the model may be a ratio of the number of inactive network parameters in the model to the initial number of network parameters in the corresponding fully-connected model, where the inactive network parameters are network parameters corresponding to two neurons that are not connected, that is, empty network parameters. The higher the sparsity of the model, the smaller the number of actual network parameters in the model.
In some embodiments, the sparsity of the generator is greater than the sparsity of the discriminator. The generator with high sparsity is obtained through training of the discriminator with known sparsity, the processing accuracy of the image generator obtained through training is guaranteed, meanwhile, the number of network parameters in the image generator is reduced, the storage space and the calculation amount of the obtained image generator are reduced, and the occupation of the storage space and the calculation capacity of the image generator on an actual application platform is reduced.
The initial generation confrontation network model comprises an initial generator and an initial discriminator, wherein the initial generator is created based on the sparsity of the generator, and the initial discriminator is created based on the sparsity of the discriminator. Specifically, connections among network layers in the generator are randomly sampled based on sparsity of the generator, so that the connections obtained by random sampling meet sparsity of the generator. In a similar way, the connection of each network layer in the discriminator is randomly sampled based on the sparsity of the discriminator, so that the connection obtained by random sampling meets the sparsity of the discriminator, and meanwhile, the discriminator generated by random sampling meets the connectivity of the model.
And acquiring sample data, and performing iterative training on the initially generated confrontation network model based on the sample data. Wherein the sample data is determined based on an application scenario for which the countermeasure network model was initially generated. In this embodiment, the application scenario for initially generating the countermeasure network model may include, but is not limited to, image enhancement, image style migration, image super resolution, image denoising, and the like, and correspondingly, the image generator obtained by iteratively training the initially generated countermeasure network model through sample data may include, but is not limited to, an enhanced image generator, a stylized image generator, a super resolution image generator, a denoising image generator, and the like. Illustratively, if the application scene is image enhancement, the sample data comprises an unenhanced image and an enhanced image; or, if the application scene is image style migration, the sample data comprises a stylized image and a basic image; or, if the application scene is image super resolution, the sample data comprises a high resolution image and a low resolution image; or, the application scene is image denoising, and the sample image comprises a noisy image and a non-noisy image.
Corresponding sample data is obtained according to an application scene of the initially generated countermeasure network model, iterative training is carried out on the initially generated countermeasure network model, and an image generator meeting the application scene is obtained.
The iterative training process for initially generating the antagonistic network model may be: inputting an unprocessed image (e.g., an unenhanced image, a low-resolution image, a noisy image, or an un-stylized base image) in a sample image into an initial generator, obtaining a predicted image by the initial generator, inputting the predicted image or a standard image (e.g., an enhanced image, a high-resolution image, an un-noisy image, or a stylized image) into an initial discriminator, obtaining an authenticity discrimination result for the input image by the initial discriminator, generating a loss function based on the authenticity discrimination result and a type of the input image, performing network parameter adjustment on an initially-generated countermeasure network model based on the loss function, wherein the network parameters comprise the network parameters in the initial generator and the network parameters in the initial discriminator, and the training process is iteratively executed to determine the generator meeting the training condition as the target image generator.
In the training process, the training object initially generates network parameters in the confrontation network model, namely connection weights corresponding to the connections in the initial generator and the initial discriminator, and does not adjust the connections in the initial generator and the initial discriminator. Due to the fact that the sparsity of the initial generator is large, in order to avoid the problem that the accuracy of an image generator obtained by training is low due to the randomness of connection of the randomly generated initial generator, in the training process of the initially generated confrontation network model, the connection relation of the initial generator is optimized to remove the connection with low importance in the generator, and the importance of the connection in the image generator obtained by training is guaranteed.
And in the iterative training process, determining whether a model updating condition is met in real time, if so, interrupting the iterative training, executing connection updating of the generator, and continuing to execute the iterative training on the updated generator after the connection updating of the generator is completed, otherwise, continuing to execute the iterative training process. The model update condition may be a preset iteration step length or a preset iteration duration, and the preset iteration step length may be 2000 iteration step lengths, for example. The model updating condition may be set according to a training requirement, and for example, the preset iteration step may be determined based on a total training step, may be 10% of the preset iteration step, and the like.
The updating of the generator connection may be determining the importance of the connection in the current generator, determining the connection to be rejected and the reserved connection based on the importance, updating the connection to be rejected, specifically, determining the newly added connections in the corresponding number, replacing the connection to be rejected, and implementing the updating of the generator.
Optionally, the updating the connection relationship of each network layer in the generator of the current iteration includes: determining connections to be updated based on connection parameters corresponding to the current connections in the generator of the current iteration, and eliminating the connections to be updated in the generator of the current iteration; and for the generator after the connection to be updated is removed, randomly sampling in the inactivated connection, determining the newly added connection, and setting the connection parameters of the newly added connection as initial connection parameters to obtain an updated generator.
In this embodiment, the importance of the connection may be determined based on a connection parameter corresponding to each current connection in the generator of the current iteration, where the connection parameter corresponding to each connection includes a connection weight, and the size of the connection weight represents the importance of the corresponding connection, where the connection weight may be a positive number or a negative number, and the greater the value (absolute value) of the connection weight, the greater the importance.
And judging the connection parameters of each connection based on the judgment rule of the connection parameters so as to determine the connection with smaller importance, namely the connection to be updated. Optionally, the determination rule of the connection parameter may be that determination is performed based on a determination threshold, and the connection corresponding to the connection parameter that meets the determination threshold is determined as the connection to be updated. Optionally, the determination rule of the connection parameter may be a ranking of the connection parameters, and the connection to be updated is determined based on the ranking.
In some embodiments, determining the connections to be updated based on the connection parameters corresponding to the current connections in the generator of the current iteration includes: and sequencing each connection based on the absolute value of the connection weight of each connection, and determining the connection in the corresponding sequencing range in the sequencing as the connection to be updated based on the expansion rate. Specifically, the absolute values of the connection weights of the connections may be sorted in size, and from the tail end of the sorting to the front, the connection satisfying the sorting range corresponding to the expansion rate is determined as the connection to be updated. The expansion rate may be a ratio of the connections to be updated, and is a numerical value between 0 and 100%, for example, the expansion rate may be 50%, and accordingly, the connection weights in the last 50% of the ordering are determined as the connections to be updated by performing size ordering based on the absolute values of the connection weights.
In some embodiments, the expansion ratio is a fixed value, such as 50% or 60%, during the overall training of the image generator. In some embodiments, during the whole training process of the image generator, the expansion rate may be changed with the number of generator updates, and for example, the expansion rate may gradually decrease with the number of generator updates.
And removing the connections to be updated from the current iteration generator, and selecting new connections from the connections which are not activated, wherein the number of the new connections is the same as that of the connections to be updated so as to meet the sparsity of the generator. The rejected connection to be updated is used as an inactivated connection, and correspondingly, the newly added connection and the connection to be updated can be overlapped. The new connection may be randomly sampled among the inactive connections distributed among the network layers.
Exemplarily, referring to fig. 2, fig. 2 is a schematic diagram of an update process of a generator according to an embodiment of the present invention. In fig. 2, the sparsity of the generator is greater than that of the discriminator, and in the iterative training, the connection corresponding to the dotted line in the generator is determined as the connection to be updated, the connection to be updated is eliminated, and the newly added connection is determined to form the updated generator.
In the embodiment, the mode of continuously adding the newly added connection into the generator is adopted, so that the total number of training parameters in the generator is increased, the problem of unbalanced sparsity of the initial generator and the initial discriminator is solved, and the smooth execution of the training process is ensured.
For the new connection, network parameters are initialized, and the connection parameters of the new connection are set as preset parameters, for example, the preset parameters may be 0 or 0.5, and the like, which is not limited herein. In some embodiments, each connection may be set with different initialization parameters, and when the connection is activated, that is, when the connection is switched from an inactive state to an active state, the initialization parameter corresponding to the newly added connection is called, so as to perform initialization setting on the newly added connection, so as to accelerate the training efficiency of the network parameters.
And forming an updated generation countermeasure network model based on the updated generator and the discriminator of the current iteration, and continuing iterative training on the updated generation countermeasure network model until an ending condition is met. Wherein the end condition may be any one of: the training times of the generated countermeasure network model meet the preset times, the training process of the generated countermeasure network model reaches a convergence state, and the training precision of the generated countermeasure network model reaches a preset precision threshold.
According to the technical scheme of the embodiment, an initially generated confrontation network model is constructed based on the sparsity of the generator and the sparsity of the discriminator, in the process of performing iterative training on the initially generated confrontation network model, connection in the initial generator is updated and optimized, the updated generated confrontation network model is continuously subjected to iterative training until the end condition is met, and the target image generator is obtained. By updating the connection with low importance in the training process, the pruning processing of the connection is realized, and the image generator giving consideration to both processing precision and sparsity is obtained after the training is finished. Meanwhile, the sparsity of the generator is kept unchanged, so that the calculated amount in the training process is reduced, and the training efficiency is improved due to the fact that the training parameters are few.
On the basis of the above embodiments, fig. 3 is a flowchart illustrating another training method for an image generator according to an embodiment of the present invention. Optionally, the generating an initial generation confrontation network model based on sparsity of the generator and the discriminator includes: randomly sampling the connection between network layers in the generator based on the sparsity of the generator, determining a first connection relation, and generating an initial generator based on the first connection relation; randomly sampling connections between network layers in the discriminator based on sparsity of the discriminator, determining a second connection relation, generating an initial discriminator based on the second connection relation, and combining the initial generator and the initial discriminator to initially generate a confrontation network model. Specifically, the method specifically comprises the following steps:
s210, randomly sampling the connection between the network layers in the generator based on the sparsity of the generator, determining a first connection relation, and generating an initial generator based on the first connection relation.
S220, randomly sampling connections among network layers in the discriminator based on sparsity of the discriminator, determining a second connection relation, and generating an initial discriminator based on the second connection relation, wherein the initial generator and the initial discriminator form an initial generation confrontation network model.
And S230, performing iterative training on the initially generated confrontation network model based on sample data, and executing the step S240 under the condition that a model updating condition is met.
S240, updating the connection relation of each network layer in the generator of the current iteration, continuing the iterative training of the step S230 on the updated generator and the discriminator of the current iteration until an end condition is met, and determining the generator after the training as a target image generator.
In this embodiment, the generator and the discriminator may include a plurality of network layers, adjacent network layers are connected to each other for information transmission, each network layer includes at least one neuron, the connection between different network layers is realized through the connection between neurons in each network layer, and different neurons may process information output by a neuron in a previous network layer, for example, convolution processing, pooling processing, and the like, which is not limited to this, and may be determined according to functions of the network layers.
In some embodiments, the generator and discriminator infrastructure may be invoked directly, the infrastructure including a plurality of network layers, each network layer including a plurality of neurons. Alternatively, the infrastructure of generators and discriminators of different depths may be invoked as required. And performing connected random sampling on the basic structure of the called generator and the basic structure of the discriminator to obtain an initial generator and an initial discriminator. It should be noted that steps S210 and S220 may be executed sequentially or synchronously, which is not limited to this.
In some embodiments, randomly sampling connections between network layers in the generator based on sparsity of the generator, determining a first connection relationship includes: acquiring a first infrastructure parameter of the generator, wherein the first infrastructure parameter comprises the number of network layers of the generator and the number of neurons of each network layer; and determining a first connection probability of each network layer in the generator based on one or more of the sparsity of the generator and the first infrastructure parameters, and randomly sampling the connection between corresponding network layers based on the first connection probability of each network layer to determine a first connection relation.
The first infrastructure parameters of the generator may be the structural parameters of a corresponding dense model of the generator, wherein the dense model may be a fully connected model.
In some embodiments, a model database may be created in advance, and the database may store the basic structure parameters of a plurality of generators, wherein the basic structure parameters of the plurality of generators may be the basic structure parameters corresponding to generators with different depths and/or widths. The model database may be stored in the cloud, and a data request may be sent to the model database in the cloud, where the data request may include one or more of a network depth and a width of the generator. And receiving the infrastructure parameters fed back by the model database of the cloud. The model database may be stored locally, and corresponding infrastructure parameters may be called from the model database according to the building requirements of the model.
The model database is an expandable database, receives the basic structure parameters uploaded by the user, and after the uploaded basic structure parameters are matched with the stored basic structure parameters, the uploaded basic structure parameters are stored under the condition that the uploaded basic structure parameters are determined to be newly added basic structure parameters, so that the subsequent calling of the user is facilitated.
Optionally, the infrastructure parameters of the generator include the number of network layers in the generator and the number of neurons in each network layer. The type of each network layer in the infrastructure parameters may be preset.
The initial generator is constructed by a first connection relation obtained by randomly sampling connection based on a first connection probability of a network layer, wherein the first connection probability of any network layer is the first connection probability between any network layer and a previous network layer. The first connection probability satisfies:
Figure BDA0003344737500000121
wherein N is the initial parameter of the network parameter in the generator, s is the sparsity of the generator, NlNumber of neurons in layer of layer l, nl-1Is the neuron number of the l-1 layer network layer, and P (l) is the first connection probability between the l-1 layer network layer and the l-1 layer network layer.
In some embodiments, the first connection probability between different network layers may be the same. Optionally, determining the first connection probability of each network layer in the generator based on one or more of the sparsity of the generator and the first infrastructure parameter includes: the sparsity of the generator is determined as the first connection probability of each network layer, illustratively, the sparsity of the generator is 50%, the first connection probability between each network layer is 50%, and the connection between each network layer is collected and sampled based on a uniform distribution mode.
In some embodiments, the first connection probabilities between different network layers may be different, and since the number of neurons in different network layers in the generator may be different, the initial number of network parameters between different network layers is different, in order to ensure the connectivity of the generator, the connection probability between network layers with fewer network parameters is increased, in order to ensure the overall sparsity of the generator, the connection probability between network layers with more network parameters is reduced, that is, the connection probability is inversely related to the initial number of network parameters between network layers.
Optionally, determining the first connection probability of each network layer in the generator based on one or more of the sparsity of the generator and the first infrastructure parameter includes: determining a first probability parameter based on the sparsity of the generator and the total number of connections of each network layer; for any network layer, determining a first connection probability for the network layer based on the number of neurons of the network layer, the number of neurons of a previous network layer, and a first probability parameter.
The first probability parameter is a parameter used for calculating connection probability between network layers in the generator and is determined based on the number of the network layers in the generator, the number of neurons in each network layer and sparsity. Specifically, the initial number of network parameters in the generator is determined based on the number of network layers in the generator and the number of neurons in each network layer, and the first probability parameter is determined based on the target number of network parameters in the generator and the number of neurons in each network layer, and specifically, the first probability parameter may be calculated based on the following formula:
Figure BDA0003344737500000131
where σ is a first probability parameter, N is an initial parameter of a network parameter in the generator, s is a sparsity of the generator, N is a first probability parameterlTo generate the number of neurons in the layer I network layer, nl-1To generate the number of neurons in the l-1 layer network layer,
in some embodiments, for any network layer in a generator, determining a first connection probability for the network layer based on a neuron number of the network layer, the neuron number of a previous network layer, and a first probability parameter may comprise: determining the number of neurons in the network layer,The sum of the number of neurons of the preceding network layer and the ratio of the sum of the number of neurons of the network layer, the number of neurons of the preceding network layer multiplied by the first probability parameter and the number multiplied by the number are determined as the first connection probability of the network layer. Illustratively, the determination is based on the following formula:
Figure BDA0003344737500000132
in some embodiments, for any network layer in a generator, determining a first connection probability for the network layer based on a neuron number of the network layer, the neuron number of a previous network layer, and a first probability parameter may comprise: determining a ratio of the number of neurons of a network layer, the number of said neurons of a preceding network layer and the product of the number of neurons of a network layer and said number of neurons of a preceding network layer, determining a first probability of connection of said network layer based on the product of the ratio and a first probability parameter. Illustratively, it is calculated based on the following formula:
Figure BDA0003344737500000141
on the basis of the foregoing embodiment, in order to avoid the situation that the first connection probability determined based on the foregoing manner is greater than 1, optionally, determining the first connection probability of the network layer based on the number of neurons of the network layer, the number of neurons of a previous network layer, and the first probability parameter includes: determining a candidate probability based on the number of neurons of the network layer, the number of neurons of a previous network layer, and a first probability parameter; determining the minimum of the candidate probability and 1 as a first connection probability.
Specifically, the candidate probability is determined according to the following formula:
Figure BDA0003344737500000142
wherein σ is a first probability parameter, nlIs the number of neurons in the l-th network layer, nl-1The number of neurons in the l-1 layer network layer.
For the generator, the connection relation of the corresponding adjacent network layer is determined based on the connection probability of each network layer, for example, the connection between the corresponding adjacent network layers is randomly sampled based on the connection probability, so as to obtain the first connection relation between the adjacent network layers. The sampled connection is activated and the non-sampled connection is in an inactive state. Optionally, a connection matrix is formed based on random sampling of connections between network layers in the generator, where sampled connections in the connection matrix are set to 1, and non-sampled connections in the connection matrix are set to 0, so as to characterize the first connection relationship of each network layer in the generator. And connecting the network layers through the connection relation of the network layers in the generator to form the initial generator.
The process of creating the initial discriminator is the same as the process of creating the initial generator, and optionally, the randomly sampling the connections between the network layers in the discriminator based on the sparsity of the discriminator to determine the second connection relationship includes: and determining a second connection probability of each network layer in the discriminator based on one or more of the sparsity of the discriminator and the second infrastructure parameters, randomly sampling the connection between the corresponding network layers based on the second connection probability of each network layer, and determining a second connection relation. Second connection probability satisfies
Figure BDA0003344737500000151
Wherein N is the initial parameter of the network parameter in the discriminator, s is the sparsity of the discriminator, NlNumber of neurons in layer I network layer in discriminator, nl-1P (l) is the second probability of connection between the l-layer network layer and the l-1 layer network layer.
The second connection probabilities between the network layers in the discriminator may be the same or different. Optionally, determining a second connection probability of each network layer in the discriminator based on one or more of the sparsity of the discriminator and the second infrastructure parameter includes: and determining the sparsity of the discriminator as a second connection probability of each network layer. Or, optionally, determining a second connection probability of each network layer in the discriminator based on one or more of the sparsity of the discriminator and the second infrastructure parameter, including: determining a second probability parameter based on the sparsity of the discriminator and the total number of connections of each network layer; for any network layer, determining a second connection probability for the network layer based on the number of neurons of the network layer, the sparsity of the discriminator of a previous network layer, and a second probability parameter.
And randomly sampling the connection between the network layers in the discriminator based on the second connection probability to obtain a second connection relation of the discriminator, and constructing the initial discriminator based on the second connection relation.
On the basis of the above embodiment, after the invoking of the infrastructure parameters of the generator or the invoking of the infrastructure parameters of the discriminator, the method further includes visually displaying the invoked infrastructure parameters, wherein the visual display may include a graphic display and a text display. The graphic display can be that each network layer and the neurons of each network layer in the generator form a model schematic graph of the generator according to the basic structure parameters of the generator, the graph is displayed, so that a user can visually know the model structure of the generator to be constructed, and the network layers and the neurons of the network layers in the discriminator form the model schematic graph of the discriminator according to the basic structure parameters of the discriminator, the graph is displayed, so that the user can visually know the model structure of the discriminator to be constructed.
The text display may be displaying the infrastructure parameters of the generator and/or the infrastructure parameters of the discriminator in text form, and optionally, for the generator or the discriminator, the content of the text display includes the number of network layers in the infrastructure parameters and the number of neurons in each network layer, and associated parameters determined based on the infrastructure parameters, wherein the associated parameters include, but are not limited to, the initial number of network parameters in the model, the actual number of network parameters in the actual constructed model (the generator and/or the discriminator), the storage space of the actual constructed model, and the training calculation amount of the actual constructed model, wherein the actual number of network parameters in the actual constructed model may be determined based on the calculation of the initial number of network parameters and the sparsity currently selected, for example, may be the calculation of the product of the initial number of network parameters and the sparsity, the storage space and the training calculated amount of the actually constructed model can be obtained by estimating the actual number of the network parameters based on the history constructed model and the corresponding storage space and training calculated amount, illustratively, the actual number of the network parameters of the actually constructed model can be matched with the actual number of the network parameters of the history constructed model, and the storage space and the training calculated amount of the history model which are successfully matched are used as the storage space and the training calculated amount of the model to be constructed; for example, a variation curve of the number of network parameters and the storage space and a variation curve of the number of network parameters and the training calculated amount may be formed based on the actual number of network parameters, the storage space and the training calculated amount of the historical model, respectively, and the corresponding storage space and the training calculated amount may be determined in the curves based on the actual number of network parameters of the actual constructed model. Through the text display mode, a user can conveniently and visually know the relevant parameters of the actually constructed model through the information displayed by the text.
Optionally, a user adjustment instruction for the called infrastructure parameters is received to update the infrastructure parameters of the generator and/or discriminator. The adjustment instruction for the infrastructure parameters includes, but is not limited to, an instruction to add or delete a network layer of the generator or the discriminator, an adjustment instruction for neuron data in any network layer of the generator or the discriminator, and the like. Accordingly, the visual display of the infrastructure parameters is updated while the infrastructure parameters are updated.
According to the technical scheme provided by the embodiment, the connection between the network layers in the generator and the discriminator is randomly sampled respectively through the obtained sparsity of the generator and the discriminator to construct the initial generator and the initial discriminator, so that the constructed generation confrontation network model meets the requirement of sparsity before the training of the generation confrontation network model, the number of network parameters in the generation confrontation network model is reduced, the storage space and the calculated amount are correspondingly reduced in the storage process and the training process of the generation confrontation network model, and the training process of the generation confrontation network model is further simplified. Meanwhile, the connection in the generation of the confrontation network model is determined by randomly acquiring the connection probability, so that the generalization of the generation of the confrontation network model is improved.
On the basis of the above embodiments, the embodiments of the present invention also provide a preferred example. Obtaining initialization parameters for generating the countermeasure network, including the generator infrastructure parameters and the generator sparsity sgThe parameters of the base structure of the discriminator, the sparsity s of the discriminatordThe time interval Δ t over which the parameters are extended (i.e. the model update condition, e.g. a preset step size), the extension rate ρ, where the sparsity s of the discriminatordSparsity s less than the generatorg. According to the initial generation of the confrontation network model by initializing the sparse parameter component, specifically, sampling the connection of each layer by a method of sampling uniform distribution, a sparse model can be obtained. The sparsity of each layer of the sparse model obtained after initialization is the same.
The initially generated confrontation network model is iteratively trained, for example, the initially generated confrontation network model may be trained based on an adam (adaptive motion estimation) algorithm. After training the initially generated antagonistic network model for a period of time, satisfying the model update conditions, e.g. 2000 training steps, the generator is updated on the connection, the unimportant parameters are pruned from the generator, and then the same number of parameters are redistributed to the current generator. The purpose of this is to allow the existing sparse connected structure to be optimized without increasing the number of parameters of the sparse generator. Specifically, the connection parameters with small values in the connection parameters are removed, that is, the connections corresponding to the connection parameters with small values in the connection parameters are removed. For exemplary purposes, see the formula
Figure BDA0003344737500000181
Specifically, the absolute values of the connection parameters in the current generator are sorted, and retention is determined based on the expansion ratio ρAnd determining the connection to be updated according to the connection parameters to be eliminated and the connection parameters to be eliminated.
Figure BDA0003344737500000182
As reserved connection parameters.
After rejecting the connections to be updated, randomly determining the newly added connections with the same quantity, and simultaneously determining the connection parameters of each newly added connection, namely determining the connection parameters with the same quantity. For example, see the formula:
Figure BDA0003344737500000183
wherein, thetat+1For updating the connection parameters in the post generator,
Figure BDA0003344737500000184
the parameters which are not activated in the current generator are the parameters which are not activated and comprise the connection parameters which are eliminated.
And continuously carrying out iterative training on the updated generation countermeasure network model formed by the updated generator and the discriminator of the current iteration, and carrying out connection updating on the generator again when the next model updating condition is met until the ending condition is met to obtain the image generator after the training is finished, thereby realizing the end-to-end training of the generation countermeasure network model of a very sparse generator and a dense discriminator.
By directly training a sparse generative confrontation network model, the number of parameters in the model and the training calculation amount are reduced. Meanwhile, the expression capacity of the sparse generator can be improved on the premise of not increasing the number of parameters by generator parameter expansion, so that the problem of unbalanced sparsity of the generator and the discriminator in the network model is solved by balanced sparse generation.
Fig. 4 is a flowchart of an image generating method according to an embodiment of the present invention, where this embodiment is applicable to a case where an image generator generates an image, and the method may be executed by an image generating apparatus according to an embodiment of the present invention, where the image generating apparatus may be implemented by software and/or hardware, and the image generating apparatus may be configured on an electronic computing device, and specifically includes the following steps:
and S310, acquiring an image to be processed.
And S310, inputting the image to be processed into a pre-trained image generator to obtain a target image output by the image generator, wherein the image generator is obtained by training based on the training method of the image generator provided by any embodiment.
In this embodiment, the corresponding to-be-processed image may be called based on a processing mode (or an application scene) of the to-be-processed image, where the processing mode of the to-be-processed image includes, but is not limited to, image stylization migration, image enhancement, image denoising, image super resolution, image compression, and the like.
Correspondingly, the image generator is an enhanced image generator, and the target image is an enhanced image corresponding to the input image; or the image generator is a stylized image generator, and the target image is a stylized image corresponding to the input image; or the image generator is a de-noised image generator, and the target image is a de-noised image corresponding to the input image; or the image generator is a super-resolution image generator, and the target image is a super-resolution image corresponding to the input image; or, the image generator is a compressed image generator, and the target image is a compressed image corresponding to the input image.
The called image generator is obtained by training based on the training method of the image generator provided by any embodiment, the image generator meets the preset sparsity, the occupied storage space is small, the required calculation amount in the processing process of the data to be processed is small, and the occupation of the storage space and the calculation power of the practical application platform is reduced.
Fig. 5 is a schematic structural diagram of an training apparatus of an image generator according to an embodiment of the present invention, where the apparatus includes:
an initial model generation module 310, configured to generate an initial generation countermeasure network model based on sparsity of the generator and the discriminator;
and the model training module 320 is configured to perform iterative training on the initially generated confrontation network model based on sample data, update the connection relationship of each network layer in the generator of the current iteration in the iterative training process under the condition that a model update condition is met, continue iterative training on the updated generator and the discriminator of the current iteration until an end condition is met, and determine the generator after training as a target image generator.
Optionally, the model training module 320 includes:
a connection to be updated determining unit, configured to determine a connection to be updated based on connection parameters corresponding to current connections in the generator of the current iteration, and eliminate the connection to be updated in the generator of the current iteration;
and the generator updating unit is used for randomly sampling the generator without the connection to be updated in the inactivated connection, determining the newly added connection, and setting the connection parameters of the newly added connection as initial connection parameters to obtain an updated generator.
Optionally, the connection parameter corresponding to the connection includes a connection weight;
the connection to be updated determining unit is configured to:
determining connections to be updated based on connection parameters corresponding to current connections in the generator of the current iteration, including:
and sequencing each connection based on the absolute value of the connection weight of each connection, and determining the connection in the corresponding sequencing range in the sequencing as the connection to be updated based on the expansion rate.
Optionally, the initial model generating module 310 includes:
the initial generator generating unit is used for randomly sampling the connection between network layers in the generator based on the sparsity of the generator, determining a first connection relation and generating an initial generator based on the first connection relation;
an initial discriminator generating unit configured to randomly sample connections between network layers in the discriminator based on sparsity of the discriminator, determine a second connection relationship, generate an initial discriminator based on the second connection relationship, the initial generator and the initial discriminator forming an initial generation countermeasure network model.
Optionally, the initial generator generating unit is configured to:
acquiring a first infrastructure parameter of the generator, wherein the first infrastructure parameter comprises the number of network layers of the generator and the number of neurons of each network layer;
determining a first connection probability of each network layer in the generator based on one or more of the sparsity of the generator and the first infrastructure parameters, randomly sampling connections among corresponding network layers based on the first connection probability of each network layer, and determining a first connection relation;
an initial discriminator generating unit for:
acquiring second infrastructure parameters of the discriminator, wherein the second infrastructure parameters comprise the number of network layers of the discriminator and the number of neurons of each network layer;
and determining a second connection probability of each network layer in the discriminator based on one or more of the sparsity of the discriminator and the second infrastructure parameters, randomly sampling the connection between the corresponding network layers based on the second connection probability of each network layer, and determining a second connection relation.
Optionally, the initial generator generating unit is configured to:
determining the sparsity of the generator as a first connection probability of each network layer; alternatively, the first and second electrodes may be,
determining a first probability parameter based on the sparsity of the generator and the total number of connections of each network layer;
for any network layer, determining a first connection probability for the network layer based on the number of neurons of the network layer, the number of neurons of a previous network layer, and a first probability parameter;
an initial discriminator generating unit for:
determining sparsity of the discriminator as a second connection probability of each network layer; alternatively, the first and second electrodes may be,
determining a second probability parameter based on the sparsity of the discriminator and the total number of connections of each network layer;
for any network layer, determining a second connection probability for the network layer based on the number of neurons of the network layer, the sparsity of the discriminator of a previous network layer, and a second probability parameter.
Optionally, the sparsity of the generator is greater than the sparsity of the discriminator.
The training device of the image generator provided by the embodiment of the invention can execute the training method of the image generator provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the training method of the image generator.
Fig. 6 is a schematic structural diagram of an image generating apparatus according to an embodiment of the present invention, where the apparatus includes:
a to-be-processed image obtaining module 410, configured to obtain a to-be-processed image;
a target image generating module 420, configured to input the image to be processed into a pre-trained image generator, so as to obtain a target image output by the image generator, where the image generator is obtained by training based on the training method of the image generator provided in any of the embodiments.
The image generation device provided by the embodiment of the invention can execute the image generation method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects for executing the image generation method.
Fig. 7 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. FIG. 7 illustrates a block diagram of an electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention. The device 12 is typically an electronic device that undertakes image classification functions.
As shown in FIG. 7, electronic device 12 is embodied in the form of a general purpose computing device. The components of electronic device 12 may include, but are not limited to: one or more processors 16, a memory device 28, and a bus 18 that connects the various system components (including the memory device 28 and the processors 16).
Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
Electronic device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by electronic device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
Storage 28 may include computer system readable media in the form of volatile Memory, such as Random Access Memory (RAM) 30 and/or cache Memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, and commonly referred to as a "hard drive"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a Compact disk-Read Only Memory (CD-ROM), a Digital Video disk (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Storage 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program 36 having a set (at least one) of program modules 26 may be stored, for example, in storage 28, such program modules 26 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may include an implementation of a gateway environment. Program modules 26 generally perform the functions and/or methodologies of the described embodiments of the invention.
Electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, camera, display 24, etc.), with one or more devices that enable a user to interact with electronic device 12, and/or with any devices (e.g., network card, modem, etc.) that enable electronic device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, electronic device 12 may communicate with one or more gateways (e.g., Local Area Network (LAN), Wide Area Network (WAN), etc.) and/or a public gateway, such as the internet, via gateway adapter 20. As shown, the gateway adapter 20 communicates with other modules of the electronic device 12 over the bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, Redundant processing units, external disk drive Arrays, disk array (RAID) systems, tape drives, and data backup storage systems, to name a few.
The processor 16 executes various functional applications and data processing, such as a training method or an image generation method of the image generator provided by the above-described embodiments of the present invention, by executing programs stored in the storage device 28.
Embodiments of the present invention provide a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements a training method or an image generation method of an image generator as provided by embodiments of the present invention.
Of course, the computer program stored on the computer-readable storage medium provided by the embodiment of the present invention is not limited to the method operations described above, and may also perform the training method of the image generator or the image generation method provided by any embodiment of the present invention.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable source code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Source code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer source code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, or the like, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The source code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of gateway, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (12)

1. A method of training an image generator, comprising:
generating an initial generation confrontation network model based on the sparsity of the generator and the discriminator;
and performing iterative training on the initially generated countermeasure network model based on sample data, updating the connection relation of each network layer in the generator of the current iteration under the condition of meeting a model updating condition in the iterative training process, continuing iterative training on the updated generator and the discriminator of the current iteration until meeting an ending condition, and determining the generator after training as a target image generator.
2. The method of claim 1, wherein updating the connection relationship of each network layer in the generator of the current iteration comprises:
determining connections to be updated based on connection parameters corresponding to the current connections in the generator of the current iteration, and eliminating the connections to be updated in the generator of the current iteration;
and for the generator after the connection to be updated is removed, randomly sampling in the inactivated connection, determining the newly added connection, and setting the connection parameters of the newly added connection as initial connection parameters to obtain an updated generator.
3. The method of claim 2, wherein the connection parameters corresponding to the connection comprise a connection weight;
determining connections to be updated based on connection parameters corresponding to current connections in the generator of the current iteration, including:
and sequencing each connection based on the absolute value of the connection weight of each connection, and determining the connection in the corresponding sequencing range in the sequencing as the connection to be updated based on the expansion rate.
4. The method of claim 1, wherein generating an initial generation countermeasure network model based on sparsity of generators and discriminators comprises:
randomly sampling the connection between network layers in the generator based on the sparsity of the generator, determining a first connection relation, and generating an initial generator based on the first connection relation;
randomly sampling connections between network layers in the discriminator based on sparsity of the discriminator, determining a second connection relation, generating an initial discriminator based on the second connection relation, and combining the initial generator and the initial discriminator to initially generate a confrontation network model.
5. The method of claim 4, wherein the randomly sampling connections between network layers in the generator based on sparsity of the generator, and determining the first connection relationship comprises:
acquiring a first infrastructure parameter of the generator, wherein the first infrastructure parameter comprises the number of network layers of the generator and the number of neurons of each network layer;
determining a first connection probability of each network layer in the generator based on one or more of the sparsity of the generator and the first infrastructure parameters, randomly sampling connections among corresponding network layers based on the first connection probability of each network layer, and determining a first connection relation;
and the number of the first and second groups,
the randomly sampling connections between network layers in the discriminator based on the sparsity of the discriminator to determine a second connection relationship, including:
acquiring second infrastructure parameters of the discriminator, wherein the second infrastructure parameters comprise the number of network layers of the discriminator and the number of neurons of each network layer;
and determining a second connection probability of each network layer in the discriminator based on one or more of the sparsity of the discriminator and the second infrastructure parameters, randomly sampling the connection between the corresponding network layers based on the second connection probability of each network layer, and determining a second connection relation.
6. The method of claim 5, wherein determining a first connection probability for each network layer in a generator based on one or more of sparsity of the generator and the first infrastructure parameter comprises:
determining the sparsity of the generator as a first connection probability of each network layer;
alternatively, the first and second electrodes may be,
determining a first probability parameter based on the sparsity of the generator and the total number of connections of each network layer;
for any network layer, determining a first connection probability for the network layer based on the number of neurons of the network layer, the number of neurons of a previous network layer, and a first probability parameter;
and determining a second connection probability for each network layer in the discriminator based on one or more of the sparsity of the discriminator and the second infrastructure parameter, comprising:
determining sparsity of the discriminator as a second connection probability of each network layer;
alternatively, the first and second electrodes may be,
determining a second probability parameter based on the sparsity of the discriminator and the total number of connections of each network layer;
for any network layer, determining a second connection probability for the network layer based on the number of neurons of the network layer, the sparsity of the discriminator of a previous network layer, and a second probability parameter.
7. The method of any of claims 1-6, wherein the sparsity of the generator is greater than the sparsity of the discriminator.
8. An image generation method, comprising:
acquiring an image to be processed;
inputting the image to be processed into a pre-trained image generator to obtain a target image output by the image generator, wherein the image generator is obtained by training based on the training method of the image generator according to any one of claims 1 to 7.
9. An training apparatus of an image generator, comprising:
the initial model generation module is used for generating an initial generation confrontation network model based on the sparsity of the generator and the discriminator;
and the model training module is used for carrying out iterative training on the initially generated confrontation network model based on sample data, updating the connection relation of each network layer in the generator of the current iteration under the condition of meeting a model updating condition in the iterative training process, continuing iterative training on the updated generator and the discriminator of the current iteration until meeting an ending condition, and determining the generator after training as a target image generator.
10. An image generation apparatus, comprising:
the image to be processed acquisition module is used for acquiring an image to be processed;
a target image generation module, configured to input the image to be processed into a pre-trained image generator, so as to obtain a target image output by the image generator, where the image generator is obtained by training based on the training method of the image generator according to any one of claims 1 to 7.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the training method of the image generator according to any of claims 1-7 or the image generation method according to claim 8 when executing the program.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a training method of an image generator according to any one of claims 1 to 7, or an image generation method according to claim 8.
CN202111318799.5A 2021-11-09 2021-11-09 Training method of image generator, image generation method and device Pending CN114037772A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111318799.5A CN114037772A (en) 2021-11-09 2021-11-09 Training method of image generator, image generation method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111318799.5A CN114037772A (en) 2021-11-09 2021-11-09 Training method of image generator, image generation method and device

Publications (1)

Publication Number Publication Date
CN114037772A true CN114037772A (en) 2022-02-11

Family

ID=80143514

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111318799.5A Pending CN114037772A (en) 2021-11-09 2021-11-09 Training method of image generator, image generation method and device

Country Status (1)

Country Link
CN (1) CN114037772A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114627342A (en) * 2022-03-03 2022-06-14 北京百度网讯科技有限公司 Training method, device and equipment of image recognition model based on sparsity

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114627342A (en) * 2022-03-03 2022-06-14 北京百度网讯科技有限公司 Training method, device and equipment of image recognition model based on sparsity

Similar Documents

Publication Publication Date Title
JP2019528502A (en) Method and apparatus for optimizing a model applicable to pattern recognition and terminal device
CN108197652B (en) Method and apparatus for generating information
CN113905391B (en) Integrated learning network traffic prediction method, system, equipment, terminal and medium
CN111488985B (en) Deep neural network model compression training method, device, equipment and medium
WO2019155064A1 (en) Data compression using jointly trained encoder, decoder, and prior neural networks
CN111105017B (en) Neural network quantization method and device and electronic equipment
CN111523640B (en) Training method and device for neural network model
CN112418292B (en) Image quality evaluation method, device, computer equipment and storage medium
CN108960425B (en) Rendering model training method, system, equipment, medium and rendering method
CN112418320B (en) Enterprise association relation identification method, device and storage medium
CN113379627A (en) Training method of image enhancement model and method for enhancing image
EP4343616A1 (en) Image classification method, model training method, device, storage medium, and computer program
CN113902010A (en) Training method of classification model, image classification method, device, equipment and medium
CN111159481B (en) Edge prediction method and device for graph data and terminal equipment
CN110633717A (en) Training method and device for target detection model
CN114065915A (en) Network model construction method, data processing method, device, medium and equipment
CN114037772A (en) Training method of image generator, image generation method and device
CN113409307A (en) Image denoising method, device and medium based on heterogeneous noise characteristics
CN113011532A (en) Classification model training method and device, computing equipment and storage medium
CN110009091B (en) Optimization of learning network in equivalence class space
CN116957006A (en) Training method, device, equipment, medium and program product of prediction model
CN116452861A (en) Target model training method and device and electronic equipment
CN116128044A (en) Model pruning method, image processing method and related devices
US20210406773A1 (en) Transforming method, training device, and inference device
WO2021218414A1 (en) Video enhancement method and apparatus, and electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination