CN112508812A - Image color cast correction method, model training method, device and equipment - Google Patents

Image color cast correction method, model training method, device and equipment Download PDF

Info

Publication number
CN112508812A
CN112508812A CN202011385880.0A CN202011385880A CN112508812A CN 112508812 A CN112508812 A CN 112508812A CN 202011385880 A CN202011385880 A CN 202011385880A CN 112508812 A CN112508812 A CN 112508812A
Authority
CN
China
Prior art keywords
image
network
corrected
correction
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011385880.0A
Other languages
Chinese (zh)
Inventor
陈扬
李启东
李志阳
周铭柯
陈进山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen Meituzhijia Technology Co Ltd
Original Assignee
Xiamen Meituzhijia Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen Meituzhijia Technology Co Ltd filed Critical Xiamen Meituzhijia Technology Co Ltd
Priority to CN202011385880.0A priority Critical patent/CN112508812A/en
Publication of CN112508812A publication Critical patent/CN112508812A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/007Dynamic range modification
    • G06T5/009Global, i.e. based on properties of the image as a whole
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Abstract

The application provides an image color cast correction method, a model training method, a device and equipment, and belongs to the technical field of image processing. The image color cast correction method comprises the following steps: acquiring an image to be corrected; inputting an image to be corrected into a color cast correction network model obtained by pre-training, and performing correction processing by the color cast correction network model to obtain a corrected image, wherein the color cast correction network model is a convolutional neural network model and comprises: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network. The color cast correction method and the color cast correction device can improve the color cast correction effect.

Description

Image color cast correction method, model training method, device and equipment
Technical Field
The application relates to the technical field of image processing, in particular to an image color cast correction method, a model training method, a device and equipment.
Background
In the field of image processing technology, due to environmental influences such as light during photographing, light of a photographed photo may be brighter or darker, and the real color of a photographed object or a scene cannot be accurately reflected, that is, color deviation occurs in images of the photo.
In order to correct such images with color deviation, the prior art usually adopts a white balance processing method to correct the images, however, the color deviation cannot be completely corrected in the conventional white balance processing, and the color deviation correction effect is poor.
Disclosure of Invention
The application aims to provide an image color cast correction method, a model training method, a device and equipment, which can improve the color cast correction effect.
The embodiment of the application is realized as follows:
in one aspect of the embodiments of the present application, a method for correcting color shift of an image is provided, including:
acquiring an image to be corrected;
inputting an image to be corrected into a color cast correction network model obtained by pre-training, and performing correction processing by the color cast correction network model to obtain a corrected image, wherein the color cast correction network model is a convolutional neural network model and comprises: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network.
Optionally, inputting the image to be corrected into a color shift correction network model obtained by pre-training, and performing correction processing by using the color shift correction network model to obtain a corrected image, including:
inputting an image to be corrected into an encoding network, and performing down-sampling processing on the image to be corrected for multiple times by the encoding network to obtain a down-sampling processing result;
inputting the downsampling processing result into a global feature extraction network, and extracting the global feature of the image to be corrected by the global feature extraction network based on the downsampling processing result;
and inputting the downsampling result and the global characteristics of the image to be corrected into a decoding network, and carrying out multiple upsampling processing by the decoding network based on the downsampling result and the global characteristics of the image to be corrected to obtain the corrected image.
Optionally, the input of the downsampling processing result into a global feature extraction network, and the global feature extraction network extracting the global feature of the image to be corrected based on the downsampling processing result includes:
and inputting the downsampling processing result into a global feature extraction network, and sequentially performing mean pooling, full connection and expansion processing on the downsampling processing result by the global feature extraction network to obtain the global feature of the image to be corrected.
Optionally, the global feature extraction network comprises: the device comprises a mean value pooling layer, a plurality of full connection layers and an expansion processing layer which are connected in sequence.
Optionally, before acquiring the image to be corrected, the method further includes:
and carrying out correction pretreatment on the initial image to obtain an image to be corrected corresponding to the initial image, wherein the correction pretreatment comprises downsampling pretreatment and scene pretreatment.
Optionally, performing correction preprocessing on the initial image to obtain an image to be corrected corresponding to the initial image, including:
performing down-sampling pretreatment on the initial image to obtain an image with a preset size, and taking the image with the preset size as an image to be corrected;
and carrying out scene preprocessing on the image with the preset size to obtain a scene fusion coefficient.
Optionally, the method for correcting color shift of an image to be corrected includes:
and adjusting the pixel value of each pixel in the corrected image by adopting the scene fusion coefficient to obtain an adjusted image.
In another aspect of the embodiments of the present application, a color cast correction network model training method is provided, including:
acquiring a training set sample image and a test set sample image;
based on the training set sample image, the test set sample image and a preset discriminator, training to obtain a color cast correction network model, wherein the color cast correction network model is a convolutional neural network model and comprises: the device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network, and a discriminator is used for discriminating the global reality and the local correction effect of the image output by a color cast correction network model.
Optionally, the discriminator comprises: global and local branches;
the global branch is used for predicting the probability that the image output by the color cast correction network model is correctly identified, and the local branch is used for carrying out image spot comparison according to the image output by the color cast correction network model and a real image corresponding to the test set sample image so as to judge the local correction effect of the color cast correction network model.
In another aspect of the embodiments of the present application, an image color shift correction apparatus is provided, including: the device comprises an acquisition module and a correction module;
the acquisition module is used for acquiring an image to be corrected;
the correction module is used for inputting an image to be corrected into a color cast correction network model obtained by pre-training, and performing correction processing on the color cast correction network model to obtain a corrected image, wherein the color cast correction network model is a convolutional neural network model and comprises: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network.
Optionally, the correction module is specifically configured to input the image to be corrected into an encoding network, and the encoding network performs downsampling processing on the image to be corrected for multiple times to obtain a downsampling processing result; inputting the downsampling processing result into a global feature extraction network, and extracting the global feature of the image to be corrected by the global feature extraction network based on the downsampling processing result; and inputting the downsampling result and the global characteristics of the image to be corrected into a decoding network, and carrying out multiple upsampling processing by the decoding network based on the downsampling result and the global characteristics of the image to be corrected to obtain the corrected image.
Optionally, the correction module is further configured to input the downsampling processing result into the global feature extraction network, and the global feature extraction network sequentially performs mean pooling, full connection, and expansion processing on the downsampling processing result to obtain the global feature of the image to be corrected.
Optionally, in the correction module, the global feature extraction network includes: the device comprises a mean value pooling layer, a plurality of full connection layers and an expansion processing layer which are connected in sequence.
Optionally, the apparatus further comprises: a preprocessing module; and the preprocessing module is used for carrying out correction preprocessing on the initial image to obtain an image to be corrected corresponding to the initial image, and the correction preprocessing comprises downsampling preprocessing and scene preprocessing.
Optionally, the preprocessing module is specifically configured to perform downsampling preprocessing on the initial image to obtain an image with a preset size, and take the image with the preset size as an image to be corrected; and carrying out scene preprocessing on the image with the preset size to obtain a scene fusion coefficient.
Optionally, the correction module is further configured to perform pixel value adjustment on each pixel in the corrected image by using the scene fusion coefficient, so as to obtain an adjusted image.
In another aspect of the embodiments of the present application, a color cast correction network model training apparatus is provided, including: the device comprises a sample acquisition module and a training module;
the sample acquisition module is used for acquiring a training set sample image and a test set sample image;
the training module is used for training to obtain a color cast correction network model based on the training set sample image, the test set sample image and a preset discriminator, the color cast correction network model is a convolutional neural network model, and the color cast correction network model comprises: the device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network, and a discriminator is used for discriminating the global reality and the local correction effect of the image output by a color cast correction network model.
Optionally, in the training module, the discriminator includes: global and local branches; the global branch is used for predicting the probability that the image output by the color cast correction network model is correctly identified, and the local branch is used for carrying out image spot comparison according to the image output by the color cast correction network model and a real image corresponding to the test set sample image so as to judge the local correction effect of the color cast correction network model.
In another aspect of the embodiments of the present application, there is provided a computer device, including: the image color cast correction method comprises a first memory and a first processor, wherein a computer program capable of running on the first processor is stored in the first memory, and when the computer program is executed by the first processor, the steps of the image color cast correction method are realized.
In another aspect of the embodiments of the present application, there is provided another computer device, including: and when the second processor executes the computer program, the steps of the color cast correction network model training method are realized.
In another aspect of the embodiments of the present application, a storage medium is provided, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the image color shift correction method and the color shift correction network model training method are implemented.
The beneficial effects of the embodiment of the application include:
in the image color shift correction method, the model training method, the device, and the apparatus provided in the embodiment of the present application, an image to be corrected is obtained and input into a color shift correction network model obtained by pre-training, and the color shift correction network model performs correction processing to obtain a corrected image, where the color shift correction network model is a convolutional neural network model and includes: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network. The global feature extraction network is used for processing the down-sampling result of the coding network, so that the situations of color lumps and excessively uneven colors in the corrected image can be prevented, the color lump effect can be effectively inhibited through the extraction of the global features, the excessively even color-cast-free result can be obtained, the correction in the color cast correction network model can keep higher robustness of the corrected result, and the color cast correction effect is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a first schematic flow chart of an image color shift correction method according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of a color shift correction network model according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating a second image color shift correction method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a global feature extraction network provided in an embodiment of the present application;
fig. 5 is a third schematic flowchart of an image color shift correction method according to an embodiment of the present application;
fig. 6 is a schematic flowchart of a color shift correction network model training method according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a discriminator according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an image color shift correction apparatus according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a color shift correction network model training apparatus according to an embodiment of the present application;
FIG. 10 is a schematic structural diagram of a computer device provided in an embodiment of the present application;
fig. 11 is a schematic structural diagram of another computer device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present application, it is noted that the terms "first", "second", "third", and the like are used merely for distinguishing between descriptions and are not intended to indicate or imply relative importance.
To facilitate a more convenient understanding of the pertinent terms of art referred to in this application, the following explanations will now be made with respect to the image processing fields that may be referred to in this application:
white balance: white balance is a way of reducing inherent colors of an object, and can realize that a camera image can accurately reflect the color condition of a shot object. The white balance is a necessary link for obtaining high-quality images in a plurality of application scenes, such as the preprocessing of a mobile device acquisition device and the retouching processing in the later stage of a studio. People may encounter the following problems when using a digital video camera for shooting: the reason why the image taken in the room of the fluorescent lamp is greenish, the scene taken under the indoor tungsten lamp light is yellowish, and the picture taken in the shadow of the sunlight is bluish is that the white balance is set. The white balance is to restore the inherent color of the object as much as possible according to the color of the object obtained at that time under different light conditions, that is, to reduce the influence of the color of the light source on the color of the object as much as possible. Different white points can be designated according to the condition of the light source in the process of color conversion, so that a corresponding correct color conversion result is obtained.
The following specifically explains a specific implementation procedure of the image color shift correction method provided in the embodiment of the present application.
Fig. 1 is a first schematic flow chart of an image color shift correction method according to an embodiment of the present application, and referring to fig. 1, the image color shift correction method includes:
s110: and acquiring an image to be corrected.
Alternatively, the image to be corrected may be an image with fixed height and width obtained by processing in a preset processing manner, and the specific height and width pixel ratios thereof may be set according to actual requirements, for example: an image of 640 (height) × 640 (width) pixels may be set as an image to be corrected.
Alternatively, the image shown on the image to be corrected may be a subject image or a person image, which is not limited herein.
S120: and inputting the image to be corrected into a color cast correction network model obtained by pre-training, and performing correction processing by the color cast correction network model to obtain a corrected image.
Wherein, the color cast correction network model is a convolution neural network model, and the color cast correction network model comprises: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network.
It should be noted that the global feature may refer to a full-size feature of the extracted image, and the range covers the full image; each image also includes local features, which may be local representations of image features.
Alternatively, the color shift correction network model may be a Neural network model obtained by pre-training, and specifically may be a CNN (Convolutional Neural Networks) model. The structure of the global feature extraction network is added on the basis of the structure of the encoding-decoding neural network, and the simulated image of the image to be corrected in the color cast correction network model can be converted from the sRGB format to the RAW format and then converted back to the sRGB format. Wherein, sRGB and RAW are two different analog image formats.
It should be noted that sRGB and RAW are descriptions about color domains of an image, an image in RAW format may refer to an original image collected by a camera sensing element without any image correction processing, and an image in sRGB (standard Red Green Blue) format may be an image in a color domain obtained by mapping an original image after a series of image processing of a camera.
In order to explain the color shift correction network model provided in the present application more clearly, the specific structure of the color shift correction network model is explained below.
Fig. 2 is a schematic structural diagram of a color shift correction network model according to an embodiment of the present disclosure, please refer to fig. 2, the color shift correction network model includes an encoding network 210, a decoding network 220, and a global feature extraction network 230, wherein the encoding network 210 may be directly connected to the decoding network 220, or may be connected to the decoding network 220 through the global feature extraction network 230.
The encoding network 210 may reversely restore the analog image in the sRGB format back to the image in the RAW format and perform color shift correction on the image in the RAW format, and after the correct white balance setting is completed, the decoding network 220 may perform decoding to generate the image in the sRGB format using the correct white balance setting, that is, the image that completes the correction process.
In the image color shift correction method provided in the embodiment of the present application, an image to be corrected may be acquired and input into a color shift correction network model obtained by pre-training, and the color shift correction network model performs correction processing to obtain a corrected image, where the color shift correction network model is a convolutional neural network model and includes: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network. The global feature extraction network is used for processing the down-sampling result of the coding network, so that the situations of color lumps and excessively uneven colors in the corrected image can be prevented, the color lump effect can be effectively inhibited through the extraction of the global features, the excessively even color-cast-free result can be obtained, the correction in the color cast correction network model can keep higher robustness of the corrected result, and the color cast correction effect is improved.
Another specific implementation of the image color shift correction method provided in the embodiment of the present application is specifically explained below.
Fig. 3 is a second flowchart of the image color shift correction method according to the embodiment of the present application, please refer to fig. 3, where an image to be corrected is input into a color shift correction network model obtained by pre-training, and the color shift correction network model performs correction processing to obtain a corrected image, including:
s310: and inputting the image to be corrected into a coding network, and performing down-sampling processing on the image to be corrected by the coding network for multiple times to obtain a down-sampling processing result.
Alternatively, after the image to be corrected is obtained, the image to be corrected may be input into an encoding network, and the encoding network may perform downsampling on the image to be corrected, where each downsampling may shorten the height and width of the image to be corrected by half, and taking the size of the image to be corrected as 640 × 640 (pixels) as an example, the result of the first downsampling is 320 × 320 (pixels). If the height of the image to be corrected is H and the width is W, the result of each down-sampling can be represented by pairs, (H/2, W/2) is the result of the first down-sampling, (H/2)n,W/2n) I.e. the result of the n-th down-sampling.
Alternatively, the specific number of downsampling may be set according to actual calculation requirements, and is not limited herein.
S320: and inputting the downsampling processing result into a global feature extraction network, and extracting the global feature of the image to be corrected by the global feature extraction network based on the downsampling processing result.
Optionally, the global feature extraction network may perform global feature extraction according to a downsampling result, so as to obtain a global feature of the image to be corrected.
S330: and inputting the downsampling result and the global characteristics of the image to be corrected into a decoding network, and carrying out multiple upsampling processing by the decoding network based on the downsampling result and the global characteristics of the image to be corrected to obtain the corrected image.
Optionally, after the global feature extraction network extracts the global feature, the decoding network may perform multiple upsampling processes based on the downsampling result and the global feature of the image to be corrected, so as to obtain a corrected image. For example, the global feature of the image to be corrected may be added to the down-sampling result of the previous stage (or added to the down-sampling result of the (n-1) th stage if the down-sampling is performed n times), the addition result may be up-sampled step by the decoding network, and the pixel size of the image to be corrected may be restored to obtain the corrected image.
Illustratively, the coding network in the color shift correction network model may perform downsampling 5 times, to obtain an intermediate image with a pixel size of (H/32, W32), and the intermediate image is subjected to upsampling step by step to the original pixel size (H, W) after being added with the global features output in the global feature extraction network, wherein the image to be corrected in the processing process includes three color channels.
The working principle and the specific implementation process of the global feature extraction network are explained by the specific structure of the global feature extraction network.
Fig. 4 is a schematic structural diagram of a global feature extraction network provided in an embodiment of the present application, please refer to fig. 4, where the global feature extraction network includes: an average pooling layer 510, a plurality of fully connected layers 520 connected in series, and an expansion processing layer 530.
The expansion processing layer 530 may be a neural network layer for performing the expansion processing, or may be a processing step for performing the expansion processing, which is not limited herein.
Optionally, the input of the downsampling processing result into a global feature extraction network, and the global feature extraction network extracting the global feature of the image to be corrected based on the downsampling processing result includes:
and inputting the downsampling processing result into a global feature extraction network, and sequentially performing mean pooling, full connection and expansion processing on the downsampling processing result by the global feature extraction network to obtain the global feature of the image to be corrected.
Optionally, the global feature extraction network provided in this embodiment of the present application may include a mean pooling layer 510, three fully-connected layers 520, and an expansion processing layer 530, where neuron parameters of the three fully-connected layers 520 are 256, 128, and 64, respectively.
Specifically, the intermediate image with the pixel size of (H/32, W32) obtained by downsampling 5 times may be added to the mean pooling layer to perform mean pooling, then the full connection layers are sequentially added to perform full connection processing, and finally the downsampling results of the expansion processing layer and the layer (H/16, W16) are subjected to expansion fusion to obtain a global feature map with the pixel size of (H/16, W16), where the feature information reflected by the global feature map is the global feature of the image to be corrected.
Alternatively, the purpose of the entire color shift correction network model is not to re-render the image into the original sRGB format image, but to generate a color shift free image using the correct white balance settings on the RAW format image. In order to multiplex the feature maps output by the coding network and extract the underlying feature information, the feature maps output by the coding network and the feature maps of the decoding network with corresponding sizes need to be connected, and the network model adopts a mode of combining the feature maps by using functions, which is different from a traditional connection mode. And the residual error layer at the outermost layer of the network uses a form of pixel-by-pixel multiplication instead of addition, and the residual error is converted into scaling so as to achieve the capability of learning scaling.
Optionally, before acquiring the image to be corrected, the method further includes:
and carrying out correction pretreatment on the initial image to obtain an image to be corrected corresponding to the initial image, wherein the correction pretreatment comprises downsampling pretreatment and scene pretreatment.
Alternatively, the initial image may be an image in which the size, format, and the like of a photograph obtained by shooting, a picture downloaded over a network, and the like are not uniform. The downsampling preprocessing may be a process of converting an initial image into an image to be corrected, and the scene preprocessing may be a process of extracting scene features in the initial image.
Next, a further specific implementation process of the image color shift correction method provided in the embodiment of the present application will be specifically explained.
Fig. 5 is a third schematic flow chart of the image color shift correction method according to the embodiment of the present application, please refer to fig. 5, where the initial image is subjected to correction preprocessing to obtain an image to be corrected corresponding to the initial image, and the method includes:
s610: and performing downsampling pretreatment on the initial image to obtain an image with a preset size, and taking the image with the preset size as an image to be corrected.
Optionally, a pyramid downsampling mode may be adopted to reduce the height and width of the initial image to obtain an image with a preset size, and the image with the preset size is used as the image to be corrected. Compared with the direct down-sampling to the pixel (640 multiplied by 640) of the image to be corrected, the method can better preserve the image details and avoid the sawtooth phenomenon.
S620: and carrying out scene preprocessing on the image with the preset size to obtain a scene fusion coefficient.
Optionally, the scene preprocessing may be performed on the image with the preset size to obtain a scene fusion coefficient k, and the scene fusion coefficient k may be used for recovering the scene.
Optionally, the method for correcting color shift of an image to be corrected includes:
and adjusting the pixel value of each pixel in the corrected image by adopting the scene fusion coefficient to obtain an adjusted image.
Optionally, the following formula may be specifically adopted for calculation:
T=I×(1-k)+O×k;
wherein T is the adjusted image, I is the initial image, O is the corrected image, k is the fusion coefficient, i.e. the scene fusion coefficient k, where k ranges from 0 to 1, and each pixel in the initial image I may be multiplied by (1-k) in the process of performing the above formula calculation; and multiplying each pixel in the corrected image by k, and adding the two results to obtain an adjusted image T.
The following explains a specific implementation process of the color shift correction network model training method provided in the embodiment of the present application.
Fig. 6 is a schematic flow chart of a color shift correction network model training method according to an embodiment of the present application, and please refer to fig. 6, the color shift correction network model training method includes:
s710: and acquiring training set sample images and test set sample images.
Alternatively, a high-definition data set or a high-definition portrait shot by a camera and a landscape (the proportion of the portrait to the landscape map is balanced, and the portrait and the landscape should be high to include a night and day scene and an indoor and outdoor scene) can be collected and divided into a training set and a test set. The training set comprises a plurality of training set sample images, and the test set comprises a plurality of test set sample images.
S720: and training to obtain a color cast correction network model based on the training set sample image, the test set sample image and a preset discriminator.
Wherein, the color cast correction network model is a convolution neural network model, and the color cast correction network model comprises: the device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network, and a discriminator is used for discriminating the global reality and the local correction effect of the image output by a color cast correction network model.
Alternatively, in order to be able to obtain an image that more closely approximates the true result, a discriminant of the impedance loss may be employed to minimize the distance between the actual light distribution and the output normal light distribution.
The specific structures of the global branch and the local branch provided in the present application are explained below by specific embodiments.
Fig. 7 is a schematic structural diagram of a discriminator provided in an embodiment of the present application, please refer to fig. 7, and optionally, the discriminator includes: a global branch 810 and a local branch 820;
the global branch 810 is used for predicting the probability that the image output by the color shift correction network model is correctly identified, and the local branch 820 is used for performing the pattern spot comparison according to the image output by the color shift correction network model and the real image corresponding to the test set sample image so as to judge the local correction effect of the color shift correction network model.
It should be noted that, because a global image discriminator often cannot process spatially varying images; for example, an input image is acquired in an indoor complex light source scene, and is affected by diffuse reflection of an indoor light source, and the correction degree of each area is different, so that the global image discriminator alone cannot provide the required adaptive capacity. In order to adaptively correct local area color cast, the discriminator comprises a global branch and a local branch, the global branch judges the reality of the corrected image, and the local branch can randomly cut a plurality of image spots from the input image for discrimination, thereby improving the local color cast correction effect.
The training process of the present application is explained below by the construction of a loss function:
in order to ensure the authenticity of the image color and contrast, a color gamut loss function of a color model (LAB) is added as supervision, the color gamut of the color model is wider, the light and shade are distinguished from the color by dark channels, a larger color range can be adjusted, and the color of the image generated by the network is controlled more accurately; in order to ensure the image perception similarity, a perception loss function is introduced; to ensure image authenticity, the distance between the color estimate of the real image and the image color distribution that generates the network output is minimized by global and local loss functions.
Wherein the global loss function is represented as:
where C denotes the global arbiter network, xrAnd xfThe distribution of real data and false data is respectively represented, and sigma represents constraint by least square loss, so that the iteration speed can be further improved. D (x)f,xr) And D (x)r,xf) In order to make the output result of the global discriminator depend on the real data x at the same timerAnd dummy data xf,(xr,xf) Finger xrRatio xfMore realistic, (x)f,xr) Finger xfRatio xrAnd is more true.Is the probability, x, that the true data is judged to be truef~PfakeIs the probability that spurious data is judged to be false.Is an expectation that spurious data is judged to be false,refers to the expectation that the real data is judged to be true.
The local loss function is expressed as:
the total loss function of the model can be obtained according to the loss function:
where α, β, γ, μ represent the weight of each loss function, respectively, L1The function of the initial loss is represented by,a global loss function is represented that is,representing the local loss function, LPercRepresenting the perceptual loss function.
In the specific training process, an Adam optimizer can be adopted, the initial learning rate of training is 0.0002, the number of iterations is 400k, and parameters can be adjusted to be α -1, β -0.5, γ -0.5, and μ -0.5 through actual training.
The following describes apparatuses, devices, and storage media for executing the image color shift correction method and the color shift correction network model training method provided in the present application, and specific implementation processes and technical effects thereof are referred to above, and will not be described again below.
Fig. 8 is a schematic structural diagram of an image color shift correction device according to an embodiment of the present application, and referring to fig. 8, the image color shift correction device includes: an acquisition module 100 and a correction module 200;
an obtaining module 100, configured to obtain an image to be corrected;
the correcting module 200 is configured to input an image to be corrected into a color shift correction network model obtained through pre-training, and perform correction processing on the color shift correction network model to obtain a corrected image, where the color shift correction network model is a convolutional neural network model and includes: the image correction device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network.
Optionally, the correcting module 200 is specifically configured to input the image to be corrected into an encoding network, and perform downsampling processing on the image to be corrected by the encoding network for multiple times to obtain a downsampling processing result; inputting the downsampling processing result into a global feature extraction network, and extracting the global feature of the image to be corrected by the global feature extraction network based on the downsampling processing result; and inputting the downsampling result and the global characteristics of the image to be corrected into a decoding network, and carrying out multiple upsampling processing by the decoding network based on the downsampling result and the global characteristics of the image to be corrected to obtain the corrected image.
Optionally, the correction module 200 is further configured to input the downsampling processing result into a global feature extraction network, and the global feature extraction network sequentially performs mean pooling, full connection and expansion processing on the downsampling processing result to obtain a global feature of the image to be corrected.
Optionally, in the correction module 200, the global feature extraction network includes: the device comprises a mean value pooling layer, a plurality of full connection layers and an expansion processing layer which are connected in sequence.
Optionally, the apparatus further comprises: a pre-processing module 300; the preprocessing module 300 is configured to perform correction preprocessing on the initial image to obtain an image to be corrected corresponding to the initial image, where the correction preprocessing includes downsampling preprocessing and scene preprocessing.
Optionally, the preprocessing module 300 is specifically configured to perform downsampling preprocessing on the initial image to obtain an image with a preset size, and use the image with the preset size as an image to be corrected; and carrying out scene preprocessing on the image with the preset size to obtain a scene fusion coefficient.
Optionally, the correcting module 200 is further configured to perform pixel value adjustment on each pixel in the corrected image by using the scene fusion coefficient, so as to obtain an adjusted image.
Fig. 9 is a schematic structural diagram of a color shift correction network model training device according to an embodiment of the present application, and referring to fig. 9, the color shift correction network model training device includes: a sample acquisition module 400 and a training module 500;
a sample obtaining module 400, configured to obtain a training set sample image and a test set sample image;
the training module 500 is configured to train to obtain a color shift correction network model based on the training set sample image, the test set sample image, and a preset discriminator, where the color shift correction network model is a convolutional neural network model, and the color shift correction network model includes: the device comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the encoding network, the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network, and a discriminator is used for discriminating the global reality and the local correction effect of the image output by a color cast correction network model.
Optionally, in the training module 500, the arbiter comprises: global and local branches; the global branch is used for predicting the probability that the image output by the color cast correction network model is correctly identified, and the local branch is used for carrying out image spot comparison according to the image output by the color cast correction network model and a real image corresponding to the test set sample image so as to judge the local correction effect of the color cast correction network model.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 10 is a schematic structural diagram of a computer device according to an embodiment of the present application, and referring to fig. 10, the computer device includes: the first memory 610 and the first processor 710, wherein the first memory 610 stores a computer program operable on the first processor 710, and the first processor 710 implements the steps of the image color shift correction method when executing the computer program.
Fig. 11 is a schematic structural diagram of another computer device provided in an embodiment of the present application, referring to fig. 11, the another computer device includes: the second memory 620 and the second processor 720, wherein the second memory 620 stores a computer program operable on the second processor 720, and the second processor 720 executes the computer program to implement the steps of the color cast correction network model training method.
In another aspect of the embodiments of the present application, a storage medium is further provided, where the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image color shift correction method and the color shift correction network model training method are implemented.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (14)

1. An image color shift correction method, comprising:
acquiring an image to be corrected;
inputting the image to be corrected into a color cast correction network model obtained by pre-training, and performing correction processing by the color cast correction network model to obtain a corrected image, wherein the color cast correction network model is a convolutional neural network model and comprises: the image correction method comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of the image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network.
2. The method of claim 1, wherein the inputting the image to be corrected into a color shift correction network model trained in advance, and performing correction processing by the color shift correction network model to obtain a corrected image comprises:
inputting the image to be corrected into the coding network, and performing down-sampling processing on the image to be corrected by the coding network for multiple times to obtain a down-sampling processing result;
inputting the downsampling processing result into the global feature extraction network, and extracting the global feature of the image to be corrected by the global feature extraction network based on the downsampling processing result;
and inputting the downsampling result and the global feature of the image to be corrected into the decoding network, and performing multiple upsampling processing on the decoding network based on the downsampling result and the global feature of the image to be corrected to obtain the corrected image.
3. The method as claimed in claim 2, wherein the inputting the downsampling processing result into the global feature extraction network, and the extracting, by the global feature extraction network, the global feature of the image to be corrected based on the downsampling processing result comprises:
and inputting the downsampling processing result into the global feature extraction network, and sequentially performing mean pooling, full connection and expansion processing on the downsampling processing result by the global feature extraction network to obtain the global feature of the image to be corrected.
4. The method of claim 3, wherein the global feature extraction network comprises: the device comprises a mean value pooling layer, a plurality of full connection layers and an expansion processing layer which are connected in sequence.
5. The method of any one of claims 1-4, wherein prior to acquiring the image to be corrected, further comprising:
and carrying out correction pretreatment on the initial image to obtain the image to be corrected corresponding to the initial image, wherein the correction pretreatment comprises down-sampling pretreatment and scene pretreatment.
6. The method according to claim 5, wherein the performing the correction preprocessing on the initial image to obtain the image to be corrected corresponding to the initial image comprises:
performing downsampling pretreatment on the initial image to obtain an image with a preset size, and taking the image with the preset size as the image to be corrected;
and carrying out scene preprocessing on the image with the preset size to obtain a scene fusion coefficient.
7. The method as claimed in claim 6, wherein the inputting the image to be corrected into a color shift correction network model trained in advance, and performing correction processing by the color shift correction network model to obtain a corrected image further comprises:
and adjusting the pixel value of each pixel in the corrected image by adopting the scene fusion coefficient to obtain an adjusted image.
8. A color cast correction network model training method is characterized by comprising the following steps:
acquiring a training set sample image and a test set sample image;
training to obtain a color cast correction network model based on the training set sample image, the test set sample image and a preset discriminator, wherein the color cast correction network model is a convolutional neural network model and comprises: the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the coding network, the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the coding network, and the discriminator is used for discriminating the global reality and the local correction effect of the image output by the color cast correction network model.
9. The method of claim 8, wherein the discriminator comprises: global and local branches;
the global branch is used for predicting the probability that the image output by the color cast correction network model is correctly identified, and the local branch is used for carrying out image spot comparison according to the image output by the color cast correction network model and a real image corresponding to the test set sample image so as to judge the local correction effect of the color cast correction network model.
10. An image color shift correction apparatus, comprising: the device comprises an acquisition module and a correction module;
the acquisition module is used for acquiring an image to be corrected;
the correction module is configured to input the image to be corrected into a color shift correction network model obtained through pre-training, and perform correction processing on the color shift correction network model to obtain a corrected image, where the color shift correction network model is a convolutional neural network model, and the color shift correction network model includes: the image correction method comprises an encoding network, a decoding network and a global feature extraction network, wherein the global feature extraction network is used for obtaining global features of the image to be corrected based on a down-sampling result of the encoding network, and the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the encoding network.
11. A color cast correction network model training device is characterized by comprising: the device comprises a sample acquisition module and a training module;
the sample acquisition module is used for acquiring a training set sample image and a test set sample image;
the training module is configured to train to obtain a color shift correction network model based on the training set sample image, the test set sample image, and a preset discriminator, where the color shift correction network model is a convolutional neural network model, and the color shift correction network model includes: the global feature extraction network is used for obtaining global features of an image to be corrected based on a down-sampling result of the coding network, the decoding network is used for carrying out decoding processing based on the global features of the image to be corrected and the down-sampling result of the coding network, and the discriminator is used for discriminating the global reality and the local correction effect of the image output by the color cast correction network model.
12. A computer device, comprising: a first memory in which a computer program is stored, the computer program being executable on the first processor, the first processor implementing the steps of the method of any of claims 1 to 7 when executing the computer program.
13. A computer device, comprising: a second memory in which a computer program is stored, the computer program being executable on the second processor, and a second processor, the second processor implementing the steps of the method according to any of the preceding claims 8 to 9 when executing the computer program.
14. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 9.
CN202011385880.0A 2020-12-01 2020-12-01 Image color cast correction method, model training method, device and equipment Pending CN112508812A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011385880.0A CN112508812A (en) 2020-12-01 2020-12-01 Image color cast correction method, model training method, device and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011385880.0A CN112508812A (en) 2020-12-01 2020-12-01 Image color cast correction method, model training method, device and equipment

Publications (1)

Publication Number Publication Date
CN112508812A true CN112508812A (en) 2021-03-16

Family

ID=74968255

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011385880.0A Pending CN112508812A (en) 2020-12-01 2020-12-01 Image color cast correction method, model training method, device and equipment

Country Status (1)

Country Link
CN (1) CN112508812A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114612347A (en) * 2022-05-11 2022-06-10 北京科技大学 Multi-module cascade underwater image enhancement method

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114612347A (en) * 2022-05-11 2022-06-10 北京科技大学 Multi-module cascade underwater image enhancement method

Similar Documents

Publication Publication Date Title
Wang et al. Gladnet: Low-light enhancement network with global awareness
CN102693538B (en) Generate the global alignment method and apparatus of high dynamic range images
CN111105376B (en) Single-exposure high-dynamic-range image generation method based on double-branch neural network
US11037278B2 (en) Systems and methods for transforming raw sensor data captured in low-light conditions to well-exposed images using neural network architectures
CN103440674A (en) Method for rapidly generating crayon special effect of digital image
CN110378845B (en) Image restoration method based on convolutional neural network under extreme conditions
CN112801904B (en) Hybrid degraded image enhancement method based on convolutional neural network
CN111292264A (en) Image high dynamic range reconstruction method based on deep learning
CN112508812A (en) Image color cast correction method, model training method, device and equipment
Ershov et al. NTIRE 2022 challenge on night photography rendering
CN112991236B (en) Image enhancement method and device based on template
CN113628143A (en) Weighted fusion image defogging method and device based on multi-scale convolution
CN113344793A (en) Image super-resolution reconstruction method, device, equipment and storage medium
Garg et al. LiCENt: Low-Light Image Enhancement Using the Light Channel of HSL
CN111986084A (en) Multi-camera low-illumination image quality enhancement method based on multi-task fusion
CN112785534A (en) Ghost-removing multi-exposure image fusion method in dynamic scene
CN108022223B (en) Tone mapping method based on logarithm mapping function blocking processing fusion
CN110717868A (en) Video high dynamic range inverse tone mapping model construction and mapping method and device
CN110225260B (en) Three-dimensional high dynamic range imaging method based on generation countermeasure network
CN113344773B (en) Single picture reconstruction HDR method based on multi-level dual feedback
CN114926381A (en) Low-illumination multi-view-angle high-quality imaging method based on image attribute fusion enhancement
Yang et al. Multi-scale Extreme Exposure Images Fusion Based on Deep Learning
Lakshmi et al. Analysis of tone mapping operators on high dynamic range images
WO2022000397A1 (en) Low-illumination image enhancement method and apparatus, and computer device
CN114067018A (en) Infrared image colorization method for generating countermeasure network based on expansion residual error

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination