CN113554083A - Multi-exposure image sample generation method and device, computer equipment and medium - Google Patents
Multi-exposure image sample generation method and device, computer equipment and medium Download PDFInfo
- Publication number
- CN113554083A CN113554083A CN202110805482.8A CN202110805482A CN113554083A CN 113554083 A CN113554083 A CN 113554083A CN 202110805482 A CN202110805482 A CN 202110805482A CN 113554083 A CN113554083 A CN 113554083A
- Authority
- CN
- China
- Prior art keywords
- image
- exposure
- feature vector
- feature
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a method and a device for generating a multi-exposure image sample, computer equipment and a medium. In one embodiment, the method comprises: inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector; multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively; and adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image. This embodiment can effectively realize the increase of many exposure image sample, and the exposure scope and the exposure difference of increase image all can be adjusted.
Description
Technical Field
The invention relates to the field of artificial intelligence. And more particularly, to a method and apparatus for generating a multi-exposure image sample, a computer device, and a medium.
Background
The model of the neural network, for example, applied to the fields of target detection, target recognition and the like, needs to be trained by utilizing a large number of image samples with different exposure levels in the early stage so as to ensure the generalization capability of the model, and the model can realize the functions of target detection, target recognition and the like for the images with different exposure levels. The inventors have found that it is difficult to obtain a large number of image samples of different exposures, and even when collected from multiple sources, it is difficult to meet the image sample requirements of thousands or even tens of thousands.
Disclosure of Invention
The invention aims to provide a method and a device for generating a multi-exposure image sample, a computer device and a medium, which are used for solving at least one of the problems in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a method for generating a multi-exposure image sample, which comprises the following steps:
inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector;
multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively; and
and adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
Optionally, the feature extractor comprises an encoding layer, a conversion layer, a decoding layer and a feature extraction layer connected in series;
the inputting the first image into a feature extractor trained by a cycle generation network architecture, and obtaining the exposure feature vector of the first image comprises:
inputting a first image into a coding layer to obtain a first feature vector;
inputting the first feature vector into a conversion layer to obtain a second feature vector after the exposure feature vector is converted;
inputting the second feature vector into a decoding layer to obtain a third feature vector; and
and inputting the third feature vector into a feature extraction layer to obtain an exposure degree feature vector.
Optionally, the feature extraction layer is a residual error network.
Optionally, the plurality of weighted values are valued at equal intervals within the set value range.
Optionally, the method further comprises: and adjusting the set value range.
Optionally, before inputting the first image into the feature extractor that has been trained by the cycle generating network architecture, the method further comprises: and training by circularly generating a network architecture to obtain the feature extractor.
Optionally, the training of the network architecture through cycle generation to obtain the feature extractor includes:
acquiring a plurality of first training images and a plurality of second training images, wherein one of the first training images and the second training images is an underexposure image, and the other one is an overexposure image;
constructing a cycle generation network architecture, wherein the cycle generation network architecture comprises a first generator, a first discriminator, a second generator and a second discriminator, the first generator comprises a feature extractor and an addition layer, the addition layer is used for adding the input and the output of the feature extractor to obtain the output of the first generator, the output of the first generator is respectively used as the input of the second discriminator and the second generator, and the output of the second generator is respectively used as the input of the first discriminator and the first generator;
and respectively inputting the plurality of first training images into the first generator and the first discriminator, and inputting the plurality of second training images into the second generator and the second discriminator so as to train the cycle generation network architecture, thereby obtaining a trained feature extractor.
A second aspect of the present invention provides a multi-exposure image sample generation apparatus comprising: the device comprises a feature extraction module, a multiplication operation module and an addition operation module;
the feature extraction module is used for inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector;
the multiplication module is used for respectively carrying out multiplication on the exposure characteristic vector and a plurality of weighted values in a set value range;
and the addition operation module is used for performing addition operation on the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
Optionally, the feature extractor comprises an encoding layer, a conversion layer, a decoding layer and a feature extraction layer connected in series;
the coding layer is used for obtaining a first feature vector according to an input first image;
the conversion layer is used for obtaining a second feature vector after the exposure degree feature vector is converted according to the input first feature vector;
the decoding layer is used for obtaining a third feature vector according to the input second feature vector; and
and the feature extraction layer is used for obtaining an exposure degree feature vector according to the input third feature vector.
Optionally, the feature extraction layer is a residual error network.
A third aspect of the present invention provides a computer apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method for generating a multi-exposure image sample as provided by the first aspect of the present invention when executing the program.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of generating a multi-exposure image sample.
The invention has the following beneficial effects:
according to the technical scheme, the method and the device for amplifying the exposure range of the multi-exposure image sample can effectively achieve the amplification of the multi-exposure image sample, and the exposure range and the exposure difference of the amplified image can be adjusted, wherein the problem that paired training samples are difficult to obtain when the feature extractor is trained can be solved by the feature extractor through the circulation generation network architecture training. Therefore, the technical scheme of the invention can greatly improve the data volume of the image samples with different exposure degrees and increase the generalization capability of the model which is obtained based on the training of the image samples with different exposure degrees and is applied to the fields of target detection, target identification and the like.
Drawings
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 illustrates an exemplary system architecture diagram in which an embodiment of the present invention may be applied.
Fig. 2 shows a flowchart of a method for generating a multi-exposure image sample according to an embodiment of the present invention.
Fig. 3 shows a schematic diagram of a cycle generation network architecture employed by an embodiment of the present invention.
FIG. 4 shows a schematic diagram of a first generator in the cycle generation network architecture shown in FIG. 3.
Fig. 5 shows a data walking diagram of a data expansion stage of the method for generating a multi-exposure image sample according to the embodiment of the present invention.
FIG. 6 shows a schematic of a plurality of image samples of different exposure levels.
Fig. 7 is a schematic structural diagram of a computer system implementing the apparatus for generating a multi-exposure image sample according to the embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the present invention, the present invention will be further described with reference to the following examples and the accompanying drawings. Similar parts in the figures are denoted by the same reference numerals. It is to be understood by persons skilled in the art that the following detailed description is illustrative and not restrictive, and is not to be taken as limiting the scope of the invention.
The model of the neural network, for example, applied to the fields of target detection, target recognition and the like, needs to be trained by utilizing a large number of image samples with different exposure levels in the early stage so as to ensure the generalization capability of the model, and the model can realize the functions of target detection, target recognition and the like for the images with different exposure levels. The inventors have found that it is difficult to obtain a large number of image samples of different exposures, and even when collected from multiple sources, it is difficult to meet the image sample requirements of thousands or even tens of thousands.
In view of this, an embodiment of the present invention provides a method for generating a multi-exposure image sample, including the following steps:
obtaining a feature extractor through the training of a cycle generation network architecture;
inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector;
multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively; and
and adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
Based on the method for generating the multi-exposure image sample provided by the embodiment, the expansion of the multi-exposure image sample can be effectively realized, the exposure range and the exposure difference of the expanded image can be adjusted, and the problem that paired training samples are difficult to obtain when the feature extractor is trained through the cyclic generation network architecture can be solved. Therefore, the method for generating the multi-exposure image sample provided by the embodiment can greatly improve the data volume of the image sample with different exposures, and increase the generalization capability of the model which is obtained based on the training of the image sample with different exposures and is applied to the fields of target detection, target identification and the like.
The method for generating a multi-exposure image sample according to this embodiment is not limited to the target included in the input first image, and may be a landscape image, a human image, or the like. The input first image is also not limited in terms of size, resolution, exposure, and other parameters, and is typically an image of normal exposure.
The method for generating a multi-exposure image sample provided in this embodiment may be implemented by a Computer device with data processing capability, specifically, the Computer device may be a Computer with data processing capability, including a Personal Computer (PC), a mini-Computer or a mainframe, or may be a server or a server cluster with data processing capability, and this embodiment is not limited thereto.
In order to facilitate understanding of the technical solution of the present embodiment, a scene of the method provided by the present embodiment in practice is described below with reference to fig. 1. Referring to fig. 1, the scenario includes a training server 10 and a generation server 20. In this embodiment, the training server 10 first obtains the feature extractor by using the training image and training through a cyclic generation network architecture. Subsequently, the generation server 20 may perform generation of the multi-exposure image sample using the feature extractor trained by the training server 10. The first image is input into the generation server 20, and image samples corresponding to a plurality of different exposures of the first image are obtained.
It should be noted that the training server 10 and the generation server 20 in fig. 1 may be two independent servers in practical application, or may be a server integrating the model training function and the image generation function. When two servers are separate, the two servers may communicate over a network that may include various types of connections, such as wired, wireless communication links, or fiber optic cables.
Next, a method of generating a multi-exposure image sample provided by the present embodiment will be described from the viewpoint of a processing apparatus having a data processing capability.
One embodiment of the present invention provides a method for generating a multi-exposure image sample, as shown in fig. 2, including the following steps:
s201, obtaining a feature extractor through the training of a cycle generation network architecture.
Next, a network structure of the cyclic generation network will be described.
A cyclic GAN is essentially two mirror symmetric Generative Networks (GANs) of defense, which form a ring network. Two GANs share two generators (generators) and each have a Discriminator (Discriminator), i.e. there are two discriminators and two generators in common. A typical advantage of Cycle GAN is that it can be trained using two sets of training pictures without pairing.For two data fields X and Y: in one aspect, the generator GX→YThe task of (2) is to convert the image Real _ X belonging to the data field X into an image Fake _ Y of the data field Y, this newly generated image Fake _ Y being passed on again to the generator G in the Cycle GANY→XTo convert back to image Rec _ X of data field X, which should be similar to Real _ X, to define a meaningful mapping that does not originally exist in the unpaired data set, i.e., the generator can retain the feature objects of the original image; on the other hand, generator GY→XThe task of (2) is to convert the image Real _ Y belonging to the data field Y into an image Fake _ X of the data field X, this newly generated image Fake _ X being passed on again to the generator G in the Cycle GANX→YTo convert back to the image Rec _ Y of data field Y.
The training of the recurrent generation network may employ a loss function of the mean square error loss:
the generator in the cyclic generation network may comprise an encoder, a converter and a decoder connected in series to generate the GY→XFor example, the encoder is configured to extract features from an input image using a convolutional neural network, e.g., compressing the image into 256 64 × 64 feature vectors; a converter, configured to convert the feature vector of the image in the data domain Y into the feature vector in the data domain X by combining the dissimilar features of the image, where the converter may use 6 layers of Resnet (residual network) modules, each of the Resnet modules is a neural network layer formed by two convolutional layers, and can achieve the goal of preserving the original image feature during conversion; and the encoder is used for completing the task of restoring the low-level features from the feature vectors by using a Deconvolution layer (Deconvolution), and finally obtaining the generated image. Generator GX→YStructure of (1) and generator GY→XSimilarly, but the generator GX→YIs a converter for converting the feature vectors of the image in the data field X into feature vectors in the data field Y.
The task of the discriminator in the recurrent generation network is and attempts to predict whether the input image is an original image or an image generated by the generator. The discriminator itself belongs to a convolutional network, and it is necessary to extract features from the image and then determine whether the extracted features belong to a particular class by adding a convolutional layer that produces a one-dimensional output.
In a possible implementation manner, in the method for generating a multi-exposure image sample provided in this embodiment, a training image set used by a feature extractor obtained through a cyclic generation network architecture training includes a first training image set and a second training image set, the first training image set includes a plurality of underexposed first training images, the second training image set includes a plurality of overexposed second training images, and it is assumed that the underexposed first training images belong to a data field X and the overexposed second training images belong to a data field Y. It should be noted that the first training image and the second training image do not need to be paired, and the targets included in the first training image and the second training image may be different.
In a possible implementation manner, as shown in fig. 3, in the method for generating a multi-exposure image sample provided in this embodiment, the loop generation network architecture includes: first generator GX→YFirst discriminator DXA second generator GY→XAnd a second discriminator DYFirst generator GX→YThe output Fake _ Y is respectively used as a second discriminator DYAnd a second generator GY→XInput of (2), a second generator GY→XThe output Fake _ X is respectively used as a first discriminator DXAnd a first generator GX→YIs input.
The loss function for the cycle generating network architecture training may employ a similar mean square error loss as the previously described cycle generating network.
During the training process, for two data fields X and Y:
the underexposed first training image Real _ X belonging to the data field X is input into a first generator GX→YFirst generator GX→YThe task of (1) is to convert an underexposed first training image Real _ X belonging to the data field X into an overexposed image Fake _ Y of the data field Y, the purpose of this process in the training being to make the first generator GX→YLearning data field Y (overexposure) features, byAfter training, the first generator GX→YLearned is the ability to convert exposure feature vectors;
the overexposed second training image Real _ Y belonging to the data field Y is input into a second generator GY→XSecond generator GY→XThe task of (1) is to convert an overexposed second training image Real _ Y belonging to the data field Y into an underexposed image Fake _ X of the data field X, the purpose of this process in the training being to make the second generator GY→XLearning data field X (underexposure) features, trained, second generator GY→XLearned is the ability to convert exposure feature vectors;
the underexposed first training image Real _ X belonging to the data field X is also input into a first discriminator DXSecond generator GY→XThe generated image Fake _ X is also input to the first discriminator DXFirst discriminator DXThe task of (a) is to discriminate that the input image is the second generator GY→XThe generated image Fake _ X is also an underexposed first training image Real _ X belonging to the data field X, and the formula is 0,1, and the loss is classified;
the overexposed second training image Real _ Y belonging to the data field Y is also input into a second discriminator DYFirst generator GX→YThe generated image Fake _ Y is also input to the second discriminator DYSecond discriminator DYIs that the first generator G is the first generator GX→YThe generated image Fake _ Y is also an overexposed second training image Real _ Y belonging to the data field Y, and the formula is 0,1, and the loss is classified;
in order to make the first generator GX→YAnd a second generator GY→XCan retain the characteristic target of the original image, and the first generator G is used in the training processX→YThe generated image Fake _ Y is also passed to the generator GY→XWith the image Rec _ X converted back into data field X, a second generator GY→XThe generated image Fake _ X is then passed to the generator GX→YTraining the first generator G with the image Rec _ Y converted back into the data field Y by calculating the loss between Rec _ Y and Real _ Y and the loss between Rec _ X and Real _ X, respectivelyX→YAnd a second generator GY→X。
In one possible implementation, as shown in FIG. 4, a first generator GX→YComprises a feature extractor 410 and an addition layer 420, wherein the addition layer 420 is used for adding the input and output of the feature extractor 410 to obtain a first generator GX→YThe output of (e.g., Fake _ Y (or Rec _ Y).
Because in the training process, the first generator GX→YIs to convert an underexposed first training image Real _ X belonging to the data field X into an overexposed image Fake _ Y of the data field Y (furthermore, a first generator GX→YFurther comprising the second generator GY→XThe generated image Fake _ X is converted back to the image Rec _ Y of the data field Y), the first generator GX→YA first generator G which learns the characteristics of the data field Y (overexposure) and is trainedX→YSince the ability to convert the exposure feature vector is learned, under the setting that an image of another data domain is to be obtained (overexposure) by adding the extracted feature vector to the original image (underexposure), the feature extractor 410 learns that the exposure feature vector of the image is converted and extracted, and the trained feature extractor 410 learns that the ability to convert and extract the exposure feature vector is to be obtained.
In one possible implementation, as shown in FIG. 4, a first generator GX→YThe feature extractor 410 of (a) comprises an encoding layer 411, a translation layer 412, a decoding layer 413 and a feature extraction layer 414 connected in series.
In the training process:
an encoding layer 411 for generating a first training image Real _ X (or a second generator G) belonging to the data field X, under-exposed according to the input features extractor 410Y→XThe generated image take _ X), obtaining a first feature vector, that is, the encoding layer 411 extracts the first feature vector from the input image through convolution operation;
a conversion layer 412, configured to obtain a second feature vector after the exposure feature vector conversion according to the input first feature vector, for example, the conversion layer 412 converts a feature vector of the image in the data field X (representing an underexposed exposure feature vector) into a feature vector in the data field Y (representing an overexposed exposure feature vector) by combining different features of the image;
the decoding layer 413 is configured to obtain a third feature vector according to the input second feature vector, for example, the decoding layer 413 obtains the third feature vector by performing a deconvolution operation on the second feature vector; and
the feature extraction layer 414 is configured to obtain an exposure feature vector according to the input third feature vector, for example, the feature extraction layer 414 extracts the exposure feature vector from the third feature vector output by the decoding layer 413 through a convolution operation.
In this implementation, the structures of the encoding layer 411, the conversion layer 412 and the decoding layer 413 are similar to the serially connected encoder, converter and decoder included in the generator in the aforementioned cyclic generation network, except that the output of the decoder of the generator in the aforementioned cyclic generation network is a generated image, while the output of the decoding layer 413 in this implementation is a feature vector, for example, the number of channels of the first training image Real _ X (or image Fake _ X) is usually 3, and the number of channels of the third feature vector output by the decoding layer 413 can be set to 128, 256, and so on, so that after the feature extraction by the feature extraction layer 414 based on convolution operation, the number of channels of the output exposure feature vector is 3, and can be compared with the underexposed first training image Real _ X (or second training image generator G) belonging to the data field X input to the feature extractor 410Y→XThe generated image take _ X) to implement the addition operation.
In one possible implementation, the feature extraction layer 414 is a Residual Network (ResNet). The residual error network has the advantages of being easy to optimize, capable of improving accuracy rate by increasing equivalent depth and the like, the residual error block in the residual error network uses jump connection, and the degradation problem of the deep network is solved through residual error learning.
In the first generator G, the first generator GX→YIncluding the feature extractor 410 and the addition layer 420, the second generator GY→XCan be directly connected in series with the loop generation networkThe encoder, converter and generator of the decoder.
In addition, the second generator G can also be usedY→XArranged in a structure comprising a feature extractor and an addition layer, or a first generator GX→YAnd a second generator GY→XAre provided as a structure including a feature extractor and an addition operation layer. The feature extractor can be obtained through the training of the cycle generation network architecture in all the modes.
S202, inputting the first image into a feature extractor which is trained by a cycle generation network architecture to obtain an exposure degree feature vector.
In one possible implementation, the feature extractor includes a coding layer, a conversion layer, a decoding layer, and a feature extraction layer connected in series;
step S202 includes:
inputting a first image into a coding layer to obtain a first feature vector;
inputting the first feature vector into a conversion layer to obtain a second feature vector after the exposure feature vector is converted;
inputting the second feature vector into a decoding layer to obtain a third feature vector; and
and inputting the third feature vector into a feature extraction layer to obtain an exposure degree feature vector.
Here, referring to the description of the training process in step S201, the feature extractor learns the capability of converting and extracting the exposure feature vector by the cycle generation network architecture training.
And S203, multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively.
S204, adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
In one possible implementation manner, the plurality of weighted values are valued at equal intervals within the set value range. In this way, a series of image samples with a uniform exposure distribution can be obtained.
In a possible implementation manner, the method for generating a multi-exposure image sample provided by this embodiment further includes: and adjusting the set value range.
For example, the setting value range may be adjusted in advance when the exposure information or the exposure level (underexposure, normal exposure, overexposure) of the first image is known; or after a series of image samples with different exposure degrees are obtained, adjusting the set value range according to the exposure degree conditions of the plurality of image samples to perform multiplication operation and subsequent addition operation again, thereby obtaining a series of image samples with different exposure degrees again.
The method for generating a multi-exposure image sample provided by this embodiment includes two stages, where step S201 may be regarded as belonging to a training stage, and steps S202 to S204 may be regarded as belonging to a data augmentation stage.
In the data augmentation stage, only the feature extractor is used for the cycle generation network architecture. As shown in FIG. 5, the input first image of unlimited target type is respectively input into an addition operator and a feature extractor trained by the cycle generation network architecture, the feature extractor trained by the cycle generation network architecture extracts an exposure feature vector A of the first image and outputs the exposure feature vector A to a multiplication operator, and the multiplication operator multiplies the exposure feature vector A of the first image with, for example [ -2,2 [ -2]Setting N weighted values w in the value range1,w2,…,wNRespectively carrying out multiplication to obtain exposure characteristic vectors A1,A2,…,ANAnd output to the adder, which adds the exposure characteristic vector A1,A2,…,ANAnd respectively carrying out addition operation with the first image to obtain N image samples with different exposure degrees corresponding to the first image.
In a specific example, the first image is input as the fourth image from the left (i.e., the image of normal exposure) among the seven images shown in fig. 6. The weighted value range is set to [ -2,2]Within the range, seven weight values are taken, and are respectively w1＝-2，w2＝-1.5，w3＝-1，w4＝0，w5＝1，w6＝1.5，w7Adding the result of multiplying the exposure characteristic vector A of the first image by different weight values to the original image of the first image to obtain 7 image samples with different exposures, wherein the weight value w is1To w7The corresponding image samples are ordered from left to right as shown in the seven images of fig. 6. When the weighted value is negative, the corresponding image sample is an underexposed image compared with the original image; under the condition that the weighted value is zero, the image sample is the original image; for the case where the weight value is positive, the corresponding image sample is an overexposed image compared to the original image.
For example, if the first image is input, for example, the second image from the left (i.e., the underexposed image) of the seven images shown in fig. 6, the exposure interval of the obtained image sample may be adjusted by shifting the value range to the right, for example, setting the value range of the weight value to [ -1,3], [ -1,4], and so on.
Another embodiment of the present invention provides a multi-exposure image sample generation apparatus, including:
the characteristic extraction module is used for inputting the first image into a characteristic extractor which is trained by a cycle generation network architecture to obtain an exposure degree characteristic vector;
the multiplication module is used for respectively carrying out multiplication on the exposure characteristic vector and a plurality of weighted values in a set value range; and
and the addition operation module is used for performing addition operation on the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
In one possible implementation manner, the feature extractor comprises an encoding layer, a conversion layer, a decoding layer and a feature extraction layer which are connected in series;
the inputting the first image into a feature extractor trained by a cycle generation network architecture, and obtaining the exposure feature vector of the first image comprises:
the coding layer is used for obtaining a first feature vector according to an input first image;
the conversion layer is used for obtaining a second feature vector after the exposure degree feature vector is converted according to the input first feature vector;
the decoding layer is used for obtaining a third feature vector according to the input second feature vector; and
and the feature extraction layer is used for obtaining an exposure degree feature vector according to the input third feature vector.
In one possible implementation, the feature extraction layer is a residual network.
It should be noted that the principle and the work flow of the apparatus for generating a multi-exposure image sample provided in this embodiment are similar to those of the method for generating a multi-exposure image sample, and reference may be made to the above description for relevant points, which is not repeated herein.
As shown in fig. 7, a computer system or a computer device suitable for implementing the generation apparatus of a multi-exposure image sample provided by the above-described embodiment includes a central processing module (CPU) that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) or a program loaded from a storage section into a Random Access Memory (RAM). In the RAM, various programs and data necessary for the operation of the computer system are also stored. The CPU, ROM, and RAM are connected thereto via a bus. An input/output (I/O) interface is also connected to the bus.
An input section including a keyboard, a mouse, and the like; an output section including a speaker and the like such as a Liquid Crystal Display (LCD); a storage section including a hard disk and the like; and a communication section including a network interface card such as a LAN card, a modem, or the like. The communication section performs communication processing via a network such as the internet. The drive is also connected to the I/O interface as needed. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive as necessary, so that a computer program read out therefrom is mounted into the storage section as necessary.
In particular, the processes described in the above flowcharts may be implemented as computer software programs according to the present embodiment. For example, the present embodiments include a computer program product comprising a computer program tangibly embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium.
The flowchart and schematic diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to the present embodiments. In this regard, each block in the flowchart or schematic diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the schematic and/or flowchart illustration, and combinations of blocks in the schematic and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the present embodiment may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises a feature extraction module, a multiplication operation module and an addition operation module. Wherein the names of the modules do not in some cases constitute a limitation of the module itself. For example, the multiplication module may also be described as a "multiplication module".
For example, the structure and function of the generation server 20 shown in fig. 1 can refer to a computer system or a computer device shown in fig. 7 and adapted to be used to implement the generation apparatus of the multi-exposure image sample provided by the above-mentioned embodiment, the computer device includes a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor implements: obtaining a feature extractor through the training of a cycle generation network architecture; inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector; multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively; and adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
For example, the structure and functions of the training server 10 shown in fig. 1 can also refer to a computer system or a computer device shown in fig. 7 and adapted to be used to implement the multi-exposure image sample generating apparatus provided in the above-mentioned embodiment, the computer device includes a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor implements: and training by circularly generating a network architecture to obtain the feature extractor.
On the other hand, the present embodiment also provides a nonvolatile computer storage medium, which may be the nonvolatile computer storage medium included in the apparatus in the foregoing embodiment, or may be a nonvolatile computer storage medium that exists separately and is not assembled into a terminal. The non-volatile computer storage medium stores one or more programs that, when executed by a device, cause the device to: obtaining a feature extractor through the training of a cycle generation network architecture; inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector; multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively; and adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
In the description of the present invention, it should be noted that the terms "upper", "lower", and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, which are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and operate, and thus, should not be construed as limiting the present invention. Unless expressly stated or limited otherwise, the terms "mounted," "connected," and "connected" are intended to be inclusive and mean, for example, that they may be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
It is further noted that, in the description of the present invention, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
It should be understood that the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention, and it will be obvious to those skilled in the art that other variations and modifications can be made on the basis of the above description, and all embodiments cannot be exhaustive, and all obvious variations and modifications belonging to the technical scheme of the present invention are within the protection scope of the present invention.
Claims (12)
1. A method of generating a multi-exposure image sample, comprising:
inputting the first image into a feature extractor which has been trained by a cycle generation network architecture to obtain an exposure degree feature vector;
multiplying the exposure characteristic vector and a plurality of weighted values in a set value range respectively; and
and adding the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
2. The method of claim 1, wherein the feature extractor comprises an encoding layer, a translation layer, a decoding layer, and a feature extraction layer connected in series;
the inputting the first image into a feature extractor trained by a cycle generation network architecture, and obtaining the exposure feature vector of the first image comprises:
inputting a first image into a coding layer to obtain a first feature vector;
inputting the first feature vector into a conversion layer to obtain a second feature vector after the exposure feature vector is converted;
inputting the second feature vector into a decoding layer to obtain a third feature vector; and
and inputting the third feature vector into a feature extraction layer to obtain an exposure degree feature vector.
3. The method of claim 2, wherein the feature extraction layer is a residual network.
4. The method of claim 1, wherein the plurality of weight values are equally spaced within the set range of values.
5. The method of claim 1, further comprising: and adjusting the set value range.
6. The method of any of claims 1-5, wherein prior to inputting the first image into the feature extractor that has been trained by the cycle generating network architecture, the method further comprises: and training by circularly generating a network architecture to obtain the feature extractor.
7. The method of claim 6, wherein training the feature extractor through the recurrent generation network architecture comprises:
acquiring a plurality of first training images and a plurality of second training images, wherein one of the first training images and the second training images is an underexposure image, and the other one is an overexposure image;
constructing a cycle generation network architecture, wherein the cycle generation network architecture comprises a first generator, a first discriminator, a second generator and a second discriminator, the first generator comprises a feature extractor and an addition layer, the addition layer is used for adding the input and the output of the feature extractor to obtain the output of the first generator, the output of the first generator is respectively used as the input of the second discriminator and the second generator, and the output of the second generator is respectively used as the input of the first discriminator and the first generator;
and respectively inputting the plurality of first training images into the first generator and the first discriminator, and inputting the plurality of second training images into the second generator and the second discriminator so as to train the cycle generation network architecture, thereby obtaining a trained feature extractor.
8. An apparatus for generating a multi-exposure image sample, comprising:
the characteristic extraction module is used for inputting the first image into a characteristic extractor which is trained by a cycle generation network architecture to obtain an exposure degree characteristic vector;
the multiplication module is used for respectively carrying out multiplication on the exposure characteristic vector and a plurality of weighted values in a set value range; and
and the addition operation module is used for performing addition operation on the multiplication result and the first image to obtain a plurality of image samples with different exposure degrees corresponding to the first image.
9. The apparatus of claim 8, wherein the feature extractor comprises an encoding layer, a translation layer, a decoding layer, and a feature extraction layer connected in series;
the coding layer is used for obtaining a first feature vector according to an input first image;
the conversion layer is used for obtaining a second feature vector after the exposure degree feature vector is converted according to the input first feature vector;
the decoding layer is used for obtaining a third feature vector according to the input second feature vector; and
and the feature extraction layer is used for obtaining an exposure degree feature vector according to the input third feature vector.
10. The apparatus of claim 9, wherein the feature extraction layer is a residual network.
11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-7 when executing the program.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110805482.8A CN113554083A (en) | 2021-07-16 | 2021-07-16 | Multi-exposure image sample generation method and device, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110805482.8A CN113554083A (en) | 2021-07-16 | 2021-07-16 | Multi-exposure image sample generation method and device, computer equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113554083A true CN113554083A (en) | 2021-10-26 |
Family
ID=78131880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110805482.8A Pending CN113554083A (en) | 2021-07-16 | 2021-07-16 | Multi-exposure image sample generation method and device, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113554083A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115578662A (en) * | 2022-11-23 | 2023-01-06 | 国网智能科技股份有限公司 | Unmanned aerial vehicle front-end image processing method, system, storage medium and equipment |
-
2021
- 2021-07-16 CN CN202110805482.8A patent/CN113554083A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115578662A (en) * | 2022-11-23 | 2023-01-06 | 国网智能科技股份有限公司 | Unmanned aerial vehicle front-end image processing method, system, storage medium and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10834415B2 (en) | Devices for compression/decompression, system, chip, and electronic device | |
| Thai et al. | Statistical model of quantized DCT coefficients: Application in the steganalysis of Jsteg algorithm | |
| CN109636721B (en) | Video super-resolution method based on countermeasure learning and attention mechanism | |
| CN111275784B (en) | Method and device for generating image | |
| CN108154080B (en) | Method for quickly tracing to source of video equipment | |
| CN112258625B (en) | Method and system for reconstructing single image to three-dimensional point cloud model based on attention mechanism | |
| WO2007075824A2 (en) | Determining intensity similarity in low-light conditions using the poisson-quantization noise model | |
| CN108038852B (en) | Image fusion quality evaluation method based on joint sparse representation model | |
| CN113554083A (en) | Multi-exposure image sample generation method and device, computer equipment and medium | |
| CN116091288A (en) | Diffusion model-based image steganography method | |
| CN115100148A (en) | Crop pest detection method based on light-weight convolutional neural network | |
| Li et al. | Multi-modality ensemble distortion for spatial steganography with dynamic cost correction | |
| CN111105364B (en) | Image restoration method based on rank one decomposition and neural network | |
| CN111542838B (en) | Quantification method and device of convolutional neural network and electronic equipment | |
| CN116798536A (en) | Training method and device for molecular multi-conformation prediction model | |
| Dheeraj et al. | Detecting Deepfakes Using Deep Learning | |
| Gaikwad | A Review on Self Learning based Methods for Real World Single Image Super Resolution | |
| Mei et al. | Deep residual refining based pseudo‐multi‐frame network for effective single image super‐resolution | |
| CN114972886A (en) | Image steganography analysis method | |
| Vila et al. | Analysis of image informativeness measures | |
| CN114419341A (en) | Convolutional neural network image identification method based on transfer learning improvement | |
| CN114387169A (en) | Image processing method and device, electronic equipment and storage medium | |
| Kasem et al. | DRCS-SR: Deep robust compressed sensing for single image super-resolution | |
| CN115689881A (en) | Image super-resolution method, system, storage medium and terminal equipment | |
| CN115103118B (en) | High dynamic range image generation method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |