CN116030365A - Model training method, apparatus, computer device, storage medium, and program product - Google Patents
Model training method, apparatus, computer device, storage medium, and program product Download PDFInfo
- Publication number
- CN116030365A CN116030365A CN202211724958.6A CN202211724958A CN116030365A CN 116030365 A CN116030365 A CN 116030365A CN 202211724958 A CN202211724958 A CN 202211724958A CN 116030365 A CN116030365 A CN 116030365A
- Authority
- CN
- China
- Prior art keywords
- image
- sample
- resolution image
- pseudo
- visible light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The present application relates to a model training method, apparatus, computer device, storage medium and program product, the method comprising: acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image; inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image; inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image; training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image; and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image. The image conversion model trained by the method can convert the low-resolution image into a clear high-resolution image.
Description
Technical Field
The present application relates to the field of artificial intelligence technology, and in particular, to a model training method, apparatus, computer device, storage medium, and program product.
Background
Electric power systems occupy a vital place in social production and daily life. The multi-view and multi-period inspection of the power system is beneficial to monitoring the running state of the power equipment, helps us to accurately know the operation and maintenance conditions of each link of the power system, and is convenient for operation and maintenance personnel to overhaul in time. The inspection is carried out on the power system, besides the fact that the image acquisition equipment on the unmanned aerial vehicle is required to be adopted to shoot the power system to acquire images in the daytime, the infrared imaging instrument is required to be adopted to acquire images of the power system through the infrared imaging technology at night.
The image obtained by unmanned aerial vehicle photographing or infrared imaging technology is a low-resolution image, the resolution is low, and the image needs to be processed. The low resolution image is processed by using a conventional model, and the obtained high resolution image is not clear enough, so that improvement is needed.
Disclosure of Invention
Based on this, it is necessary to provide a model training method, apparatus, computer device, storage medium and program product in order to solve the above technical problems.
In a first aspect, the present application provides a model training method. The method comprises the following steps:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In one embodiment, the training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image includes:
training a first countermeasure discriminator of the first countermeasure network using the pseudo visible light image and the sample visible light image;
training the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image.
In one embodiment, the training the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image includes:
training the first countermeasure generator based on a loss function from the sample low fraction image;
determining a norm loss from the first pseudo low resolution image and the sample low resolution image;
the trained first countermeasure generator is optimized with the norm loss.
In one embodiment, the training the second countering network in the image conversion model using the sample low-resolution image, the pseudo-visible image, and the sample high-resolution image includes:
fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image;
inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model to generate a pseudo high-resolution image;
training the second countermeasure network with the pseudo high resolution image and the sample high resolution image.
In one embodiment thereof, the method further comprises:
inputting the pseudo high-resolution image into a second auxiliary generator to obtain a second pseudo low-resolution image;
verifying the rationality of a second countermeasure generator of the trained second countermeasure network from the second pseudo low resolution image and the sample low resolution image.
In one embodiment, the first countermeasure generator of the first countermeasure network is a variation automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network.
In a second aspect, the present application further provides an optimization apparatus for a target detection model. The device comprises:
the image acquisition module is used for acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
the first generation module is used for inputting the sample low-resolution image into a first countermeasure generator of a first countermeasure network in the image conversion model to generate a pseudo visible light image;
the second generation module is used for inputting the pseudo visible light image into the first auxiliary generator to obtain a first pseudo low-resolution image;
the first training module is used for training the first countermeasure network by adopting the pseudo visible light image, the first pseudo low-resolution image, the sample low-resolution image and the sample visible light image;
and the second training module is used for training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
The above model training method, apparatus, computer device, storage medium and program product generate a pseudo visible light image by inputting a sample low-resolution image into a first countermeasure generator of a first countermeasure network, and inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo-resolution image. Training a first countermeasure network by adopting a pseudo visible light image, a first pseudo-resolution image, a sample low-resolution image and a sample visible light image; and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image. The image conversion model obtained by training can convert the low-resolution image into a clear high-resolution image.
Drawings
FIG. 1 is a flow chart of a model training method in one embodiment;
FIG. 2 is a flow diagram of training a first countering network in one embodiment;
FIG. 3 is a flow diagram of training a second countermeasure network in one embodiment;
FIG. 4 is a flow chart of a model training method in another embodiment
FIG. 5 is a block diagram of a model training device in one embodiment;
FIG. 6 is a block diagram of a model training device in another embodiment;
FIG. 7 is a block diagram of a model training device in yet another embodiment;
fig. 8 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, a model training method is provided, which can be applied to a terminal or a server. The terminal may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and the like. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers. The method is applied to a server for illustration, and specifically comprises the following steps:
s101, acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image.
Specifically, the image of the sample can be acquired through the image acquisition equipment on the unmanned aerial vehicle or the tower base camera to obtain a sample image set. Each sample image set may comprise a plurality of sample groups, each sample group comprising one sample low fraction image, one sample visible light image, and one sample high resolution image; the three sample images included in each sample group correspond to the same scene.
S102, inputting the sample low-resolution image into a first countermeasure generator of a first countermeasure network in the image conversion model, and generating a pseudo visible light image.
In the present embodiment, the so-called image conversion model is a model for converting a low-resolution image into a high-resolution model; optionally, the image conversion model includes two countermeasure networks; wherein the first countermeasure network is used for converting the low resolution image into a visible light image.
Specifically, the sample low-resolution image is input to a first countermeasure generator of a first countermeasure network, which generates a pseudo visible light image based on a pixel probability distribution of the sample low-resolution image. Optionally, the sample low resolution image within each sample group corresponds to one pseudo visible light image.
And S103, inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image.
In this embodiment, the first assistance generator is used for assisting in training the first countermeasure network, and further, is used for assisting in training the first countermeasure generator in the first countermeasure network.
Specifically, a first countermeasures discriminator of the first countermeasures network may be used to discriminate the pseudo visible light image corresponding to each sample group from the sample visible light discrimination image in each sample group, so as to obtain the pseudo visible light image discriminated by the first countermeasures discriminator and having a larger difference from the sample visible light image; the acquired pseudo-visible light image is input to a first auxiliary generator that generates a first pseudo-resolution image based on a pixel probability distribution of the pseudo-visible light image.
And S104, training the first countermeasure network by adopting the pseudo visible light image, the first pseudo low-resolution image, the sample low-resolution image and the sample visible light image.
Specifically, the training loss may be determined based on the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image according to a predetermined loss function, and the training loss is used to train the first countermeasure network.
S105, training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
Optionally, a second countermeasure network in the image conversion model is used to convert the visible light image into a high resolution image. The second countermeasure network includes a second countermeasure generator and a second countermeasure arbiter; further, the first countermeasure generator and the second countermeasure generator are different, and optionally, the first countermeasure generator of the first countermeasure network is a variation automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network. The variational automatic encoder is a generated network structure based on variational Bayesian inference. Unlike conventional self-encoders which describe potential space numerically, it describes the observation of potential space probabilistically. The residual network is easy to optimize, accuracy can be improved by increasing a considerable depth, and jump connection is used for the residual blocks inside the residual network, so that the gradient vanishing problem caused by increasing the depth in the deep neural network is relieved.
Specifically, according to a preset loss function, training loss can be determined based on the sample low-resolution image, the pseudo visible light image and the sample high-resolution image, and the second reactance network in the image conversion model is trained.
It will be appreciated that after the first and second countermeasure networks are trained, a trained image conversion model may be obtained. Furthermore, the target low-resolution image acquired in real time can be input into a trained image conversion model, and the cleaned high-resolution image can be obtained.
According to the model training method, the sample low-resolution image is input into the first countermeasure generator of the first countermeasure network to generate the pseudo visible light image, and the pseudo visible light image is input into the first auxiliary generator to obtain the first pseudo-resolution image. Training a first countermeasure network by adopting a pseudo visible light image, a first pseudo-resolution image, a sample low-resolution image and a sample visible light image; and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image. The image conversion model obtained by training can convert the low-resolution image into a clear high-resolution image.
In one embodiment, the step S104 is further refined, which may specifically include the following steps:
s201, training a first countermeasure discriminator of a first countermeasure network by adopting the pseudo visible light image and the sample visible light image.
Specifically, based on a preset loss function, training loss is determined according to the pseudo visible light image and the sample visible light image; the first challenge arbiter is trained using the training penalty.
Wherein, the loss function may be:
wherein log1 is close to 0, x is the visible light image of the sample, log D 1 (x) Approximately 0; at the position of
In (I)>
Representing a pseudo visible image, the true value is close to 1, since 1-1=0,
the infinite approximation is negative, i.e. the first challenge arbiter is optimized towards loss.
S202, training a first countermeasure generator by using the first pseudo low resolution image and the sample low resolution image.
Alternatively, the first countermeasure generator may be trained from the sample low fraction image based on the loss function; determining a norm loss from the first pseudo low resolution image and the sample low resolution image; the trained first countermeasure generator is optimized with a norm loss. Wherein, the definition of the loss function is as follows:
where KL represents the divergence, x represents the sample low resolution image, z represents the sample, pn is the prior distribution of the image, q v Is a parameterized distribution used to approximate the reality, typically replaced by a normal distribution, L representing the number of sampling points.
Specifically, the low-resolution image of the sample may be substituted into a loss function, and the first countermeasure generator may be optimized with the objective of minimizing the loss function; thereafter, determining a norm loss from the first pseudo low resolution image and the sample low resolution image based on a norm loss function (e.g., an L1 norm loss function); the trained network parameters of the first countermeasure generator are adjusted using the norm loss.
It will be appreciated that in this embodiment, the first countermeasure generator is optimized by introducing the first auxiliary generator such that the trained image output by the first countermeasure generator is more closely input.
In this embodiment, a specific method of training the first countermeasure discriminator of the first countermeasure network is given by training the first countermeasure discriminator with the pseudo visible light image and the sample visible light image, and training the first countermeasure generator with the first pseudo low resolution image and the sample low resolution image.
In one embodiment, the step S105 is further refined, as shown in fig. 3, and may specifically include the following steps:
s301, fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image.
Specifically, when the sample low-resolution image and the pseudo visible light image are fused, the texture of the fused image can be adjusted based on the sample low-resolution image, so that the fused image contains more texture information. The resulting fused image has the same brightness as the sample low resolution image and has the same gradient as the pseudo visible light image.
Wherein the fused image is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing a base layer image;
Texture information of the fused image is as follows:
specifically, I V Representing a pseudo visible image, b representing a label of a sample low resolution image (true image 1, no label 0), D (I v ) Representation I V Is a result of the recognition of (a).
The content loss of the fused image is as follows:
wherein H and W represent the height and width of the input image, and ||F represents the matrix norm, I f is an infrared image, I i Is a visible light image, delta is a derivative sign, meaning that the image is graded.
S302, inputting the fusion image to a second countermeasure generator of a second countermeasure network in the image conversion model, and generating a pseudo high-resolution image.
Optionally, the second countermeasure generator is mainly used for reconstructing the resolution of the fused image, and then inputting the given fused image with low resolution into the second countermeasure generator, and the second countermeasure generator restores the corresponding pseudo high resolution image based on a correlation algorithm.
S303, training the second countermeasure network by adopting the pseudo high-resolution image and the sample high-resolution image.
Specifically, the second challenge identifier in the second challenge network may be trained using the pseudo-high resolution image and the sample high resolution image in combination with a loss function as shown in equation (6) below.
Wherein, V, C respectively represent the channel size and the number of the feature spectrum, and I, I F represents the Frobenius norm. I HR Representing a sample high resolution image, I SR Representing a pseudo high resolution image.
Further, the second countermeasure generator in the second countermeasure network of the embodiment may be a pre-trained residual network, and may be directly used.
Alternatively, the pseudo high-resolution image may be input to a second auxiliary generator to obtain a second pseudo low-resolution image; the second countermeasure generator of the trained second countermeasure network is validated for rationality based on the second pseudo low resolution image and the sample low resolution image. Wherein the second auxiliary generator and the second countermeasure generator are in a network inversion relation, the network parameters are the same, and only the gradient update parameter moments are different.
Specifically, the rationality of the second countermeasure generator may be determined by comparing the similarity of the second pseudo low resolution image and the sample low resolution image, if the similarity of the second pseudo low resolution image and the sample low resolution image is greater than a threshold value, the second countermeasure generator is indicated to be unreasonable, and if the similarity of the second pseudo low resolution image and the sample low resolution image is less than the threshold value, the second countermeasure generator is indicated to be reasonable. Further, in the event that the second countermeasure generator is not rational, the second countermeasure generator may be re-optimized.
In this embodiment, a sample low-resolution image and a pseudo visible light image are fused to obtain a fused image, the fused image is input to a second countermeasure generator of a second countermeasure network in the image conversion model to generate a pseudo high-resolution image, and then the pseudo high-resolution image and the sample high-resolution image are adopted to train the second countermeasure network, thereby providing a specific method for training the second countermeasure network.
In one embodiment, as shown in fig. 4, a preferred example of a model training method is provided, and the implementation process may include:
s401, acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image.
S402, inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in the image conversion model, and generating a pseudo visible light image.
S403, inputting the pseudo visible light image to a first auxiliary generator to obtain a first pseudo low-resolution image.
S404, training a first countermeasure discriminator of the first countermeasure network by adopting the pseudo visible light image and the sample visible light image.
S405, training the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image.
S406, fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image.
S407, inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model, and generating a pseudo high resolution image.
And S408, training the second countermeasure network by adopting the pseudo high-resolution image and the sample high-resolution image.
It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides an optimization device for the target detection model, which is used for realizing the optimization method of the target detection model. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the optimizing device for one or more target detection models provided below may be referred to the limitation of the optimizing method for the target detection model hereinabove, and will not be repeated herein.
In one embodiment, as shown in the figure, there is provided a model training apparatus 1, as shown in fig. 5, comprising: an image acquisition module 10, a first generation module 20, a second generation module 30, a first training module 40, a second training module 50, wherein:
an image acquisition module 10 for acquiring a sample low-fraction image, a sample visible light image, and a sample high-resolution image;
a first generation module 20 for inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in the image conversion model to generate a pseudo visible light image;
a second generation module 30, configured to input the pseudo visible light image to the first auxiliary generator, so as to obtain a first pseudo low resolution image;
a first training module 40 for training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image;
the second training module 50 is configured to train the second reactance network in the image conversion model by using the sample low resolution image, the pseudo visible light image and the sample high resolution image.
In one embodiment, as shown in fig. 6, the first training module 40 further includes the following units:
a discriminator training unit 41 for training a first countermeasure discriminator of the first countermeasure network using the pseudo visible light image and the sample visible light image;
the generator training unit 42 is configured to train the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image.
In one embodiment, the generator training unit 42 described above is further configured to:
training the first countermeasure generator based on the loss function from the sample low fraction image; determining a norm loss from the first pseudo low resolution image and the sample low resolution image;
in one embodiment, the second training module 50 is refined on the basis of fig. 5 or 6. For example, the second training module 50 is further refined on the basis of fig. 5, and as shown in fig. 7, the second training module 50 further includes the following units:
an image fusion unit 51, configured to fuse the sample low-resolution image and the pseudo visible light image to obtain a fused image;
a high resolution generation unit 52 for inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model, generating a pseudo high resolution image;
a second training unit 53 for training the second countermeasure network using the pseudo high resolution image and the sample high resolution image.
In one embodiment, the model training apparatus 1 further includes the following modules:
the third generation module is used for inputting the pseudo high-resolution image into the second auxiliary generator to obtain a second pseudo low-resolution image;
a verification module for verifying the rationality of the second countermeasure generator of the trained second countermeasure network based on the second pseudo low resolution image and the sample low resolution image.
In one embodiment, the first countermeasure generator of the first countermeasure network is a variation automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network.
The modules in the model training apparatus described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer equipment is used for storing the initial height of the corrugated pipe to be tested and the related data such as the height, temperature and performance change rate of the standard corrugated pipe. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a model training method.
It will be appreciated by those skilled in the art that the structure shown in fig. 8 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training a first countermeasure network by adopting a pseudo visible light image, a first pseudo low-resolution image, a sample low-resolution image and a sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In one embodiment, the processor executes logic in the computer program to train the first countermeasure network using the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image, to further implement the steps of:
training a first countermeasure discriminator of a first countermeasure network by adopting a pseudo visible light image and a sample visible light image; the first countermeasure generator is trained using the first pseudo low resolution image and the sample low resolution image.
In one embodiment, the processor executes logic in the computer program that uses the first pseudo low resolution image and the sample low resolution image to train the first countermeasure generator to further implement the steps of:
training the first countermeasure generator based on the loss function from the sample low fraction image; determining a norm loss from the first pseudo low resolution image and the sample low resolution image; the trained first countermeasure generator is optimized with a norm loss.
In one embodiment, the computer program performs the logic for training the second countermeasure network in the image conversion model using the sample low resolution image, the pseudo visible light image, and the sample high resolution image, further comprising:
fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image; inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model to generate a pseudo high-resolution image; the second countermeasure network is trained using the pseudo high resolution image and the sample high resolution image.
In one embodiment, the computer program when executing logic further performs the steps of:
inputting the pseudo high-resolution image into a second auxiliary generator to obtain a second pseudo low-resolution image; the second countermeasure generator of the trained second countermeasure network is validated for rationality based on the second pseudo low resolution image and the sample low resolution image.
In one embodiment, the first countermeasure generator of the first countermeasure network is a variation automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training a first countermeasure network by adopting a pseudo visible light image, a first pseudo low-resolution image, a sample low-resolution image and a sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In one embodiment, the logic for training the first countermeasure network using the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image in the computer program when executed by the processor further performs the steps of:
training a first countermeasure discriminator of a first countermeasure network by adopting a pseudo visible light image and a sample visible light image; the first countermeasure generator is trained using the first pseudo low resolution image and the sample low resolution image.
In one embodiment, the logic for training the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image in the computer program is further implemented when executed by the processor to:
training the first countermeasure generator based on the loss function from the sample low fraction image; determining a norm loss from the first pseudo low resolution image and the sample low resolution image; the trained first countermeasure generator is optimized with a norm loss.
In one embodiment, the logic for training the second countermeasure network in the image conversion model using the sample low resolution image, the pseudo visible light image, and the sample high resolution image in the computer program is further implemented by the processor to:
fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image; inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model to generate a pseudo high-resolution image; the second countermeasure network is trained using the pseudo high resolution image and the sample high resolution image.
In one embodiment, logic in the computer program when executed by the processor further performs the steps of:
inputting the pseudo high-resolution image into a second auxiliary generator to obtain a second pseudo low-resolution image; the second countermeasure generator of the trained second countermeasure network is validated for rationality based on the second pseudo low resolution image and the sample low resolution image.
In one embodiment, the first countermeasure generator of the first countermeasure network is a variation automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training a first countermeasure network by adopting a pseudo visible light image, a first pseudo low-resolution image, a sample low-resolution image and a sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
In one embodiment, the logic for training the first countermeasure network using the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image in the computer program when executed by the processor further performs the steps of:
training a first countermeasure discriminator of a first countermeasure network by adopting a pseudo visible light image and a sample visible light image; the first countermeasure generator is trained using the first pseudo low resolution image and the sample low resolution image.
In one embodiment, the logic for training the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image in the computer program is further implemented when executed by the processor to:
training the first countermeasure generator based on the loss function from the sample low fraction image; determining a norm loss from the first pseudo low resolution image and the sample low resolution image; the trained first countermeasure generator is optimized with a norm loss.
In one embodiment, the logic for training the second countermeasure network in the image conversion model using the sample low resolution image, the pseudo visible light image, and the sample high resolution image in the computer program is further implemented by the processor to:
fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image; inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model to generate a pseudo high-resolution image; the second countermeasure network is trained using the pseudo high resolution image and the sample high resolution image.
In one embodiment, logic in the computer program when executed by the processor further performs the steps of:
inputting the pseudo high-resolution image into a second auxiliary generator to obtain a second pseudo low-resolution image; the second countermeasure generator of the trained second countermeasure network is validated for rationality based on the second pseudo low resolution image and the sample low resolution image.
In one embodiment, the first countermeasure generator of the first countermeasure network is a variation automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. A method of model training, the method comprising:
acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
inputting the sample low-resolution image to a first countermeasure generator of a first countermeasure network in an image conversion model to generate a pseudo visible light image;
inputting the pseudo visible light image into a first auxiliary generator to obtain a first pseudo low-resolution image;
training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image;
and training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
2. The method of claim 1, wherein training the first countermeasure network with the pseudo visible light image, the first pseudo low resolution image, the sample low resolution image, and the sample visible light image comprises:
training a first countermeasure discriminator of the first countermeasure network using the pseudo visible light image and the sample visible light image;
training the first countermeasure generator using the first pseudo low resolution image and the sample low resolution image.
3. The method of claim 2, wherein training the first countermeasure generator with the first pseudo low resolution image and the sample low resolution image comprises:
training the first countermeasure generator based on a loss function from the sample low fraction image;
determining a norm loss from the first pseudo low resolution image and the sample low resolution image;
the trained first countermeasure generator is optimized with the norm loss.
4. The method of claim 1, wherein training the second antagonism network in the image conversion model using the sample low resolution image, the pseudo visible image, and the sample high resolution image comprises:
fusing the sample low-resolution image and the pseudo visible light image to obtain a fused image;
inputting the fused image to a second countermeasure generator of a second countermeasure network in the image conversion model to generate a pseudo high-resolution image;
training the second countermeasure network with the pseudo high resolution image and the sample high resolution image.
5. The method according to claim 4, wherein the method further comprises:
inputting the pseudo high-resolution image into a second auxiliary generator to obtain a second pseudo low-resolution image;
verifying the rationality of a second countermeasure generator of the trained second countermeasure network from the second pseudo low resolution image and the sample low resolution image.
6. The method of any one of claims 1-5, wherein the first countermeasure generator of the first countermeasure network is a variational automatic encoder; the second countermeasure generator of the second countermeasure network is a residual network.
7. A model training apparatus, the apparatus comprising the following modules:
the image acquisition module is used for acquiring a sample low-fraction image, a sample visible light image and a sample high-resolution image;
the first generation module is used for inputting the sample low-resolution image into a first countermeasure generator of a first countermeasure network in the image conversion model to generate a pseudo visible light image;
the second generation module is used for inputting the pseudo visible light image into the first auxiliary generator to obtain a first pseudo low-resolution image;
the first training module is used for training the first countermeasure network by adopting the pseudo visible light image, the first pseudo low-resolution image, the sample low-resolution image and the sample visible light image;
and the second training module is used for training a second reactance network in the image conversion model by adopting the sample low-resolution image, the pseudo visible light image and the sample high-resolution image.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211724958.6A CN116030365A (en) | 2022-12-30 | 2022-12-30 | Model training method, apparatus, computer device, storage medium, and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211724958.6A CN116030365A (en) | 2022-12-30 | 2022-12-30 | Model training method, apparatus, computer device, storage medium, and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116030365A true CN116030365A (en) | 2023-04-28 |
Family
ID=86080685
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211724958.6A Pending CN116030365A (en) | 2022-12-30 | 2022-12-30 | Model training method, apparatus, computer device, storage medium, and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116030365A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291252A (en) * | 2023-11-27 | 2023-12-26 | 浙江华创视讯科技有限公司 | Stable video generation model training method, generation method, equipment and storage medium |
-
2022
- 2022-12-30 CN CN202211724958.6A patent/CN116030365A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291252A (en) * | 2023-11-27 | 2023-12-26 | 浙江华创视讯科技有限公司 | Stable video generation model training method, generation method, equipment and storage medium |
| CN117291252B (en) * | 2023-11-27 | 2024-02-20 | 浙江华创视讯科技有限公司 | Stable video generation model training method, generation method, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106846463B (en) | Microscopic image three-dimensional reconstruction method and system based on deep learning neural network | |
| CN110782395B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN110728295B (en) | Semi-supervised landform classification model training and landform graph construction method | |
| CN113780292A (en) | Semantic segmentation network model uncertainty quantification method based on evidence reasoning | |
| CN112102165B (en) | Light field image angular domain super-resolution system and method based on zero sample learning | |
| JP2023533907A (en) | Image processing using self-attention-based neural networks | |
| CN116030365A (en) | Model training method, apparatus, computer device, storage medium, and program product | |
| CN116503399A (en) | Insulator pollution flashover detection method based on YOLO-AFPS | |
| CN115272599A (en) | Three-dimensional semantic map construction method oriented to city information model | |
| Wang et al. | Multi‐scale network for remote sensing segmentation | |
| Ke et al. | Haze removal from a single remote sensing image based on a fully convolutional neural network | |
| CN116258756B (en) | Self-supervision monocular depth estimation method and system | |
| CN116580324A (en) | Yolov 5-based unmanned aerial vehicle ground target detection method | |
| CN116543333A (en) | Target recognition method, training method, device, equipment and medium of power system | |
| Liu et al. | CMLocate: A cross‐modal automatic visual geo‐localization framework for a natural environment without GNSS information | |
| CN115797291A (en) | Circuit terminal identification method and device, computer equipment and storage medium | |
| CN114065872A (en) | Feature reconstruction-based universal anti-disturbance construction method and system for visible light image | |
| Kori et al. | Enhanced image classification with data augmentation using position coordinates | |
| Chi et al. | Depth estimation of a single RGB image with semi-supervised two-stage regression | |
| CN117496162B (en) | Method, device and medium for removing thin cloud of infrared satellite remote sensing image | |
| CN112651330B (en) | Target object behavior detection method and device and computer equipment | |
| CN117173401B (en) | Semi-supervised medical image segmentation method and system based on cross guidance and feature level consistency dual regularization | |
| Liu et al. | A Lightweight YOLO Object Detection Algorithm Based on Bidirectional Multi‐Scale Feature Enhancement | |
| CN117909798A (en) | Electric automobile charging load scene generation method, system and equipment based on improved RAC-GAN | |
| CN117829336A (en) | Power station load prediction method, device, computer equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |