CN114119403A - Image defogging method and system based on red channel guidance - Google Patents
Image defogging method and system based on red channel guidance Download PDFInfo
- Publication number
- CN114119403A CN114119403A CN202111394646.9A CN202111394646A CN114119403A CN 114119403 A CN114119403 A CN 114119403A CN 202111394646 A CN202111394646 A CN 202111394646A CN 114119403 A CN114119403 A CN 114119403A
- Authority
- CN
- China
- Prior art keywords
- image
- defogging
- sub
- module
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012549 training Methods 0.000 claims abstract description 34
- 230000004927 fusion Effects 0.000 claims description 42
- 238000005070 sampling Methods 0.000 claims description 33
- 238000006243 chemical reaction Methods 0.000 claims description 21
- 238000012360 testing method Methods 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 18
- 238000010586 diagram Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 15
- 238000010276 construction Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 9
- 230000001131 transforming effect Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 239000011800 void material Substances 0.000 claims description 5
- 230000008447 perception Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 7
- 238000011176 pooling Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an image defogging method and system based on red channel guidance, wherein the method comprises the following steps: constructing a sample data set; constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image characteristics of the R channel image and sending the image characteristics to the main module; the main body module is used for carrying out defogging operation on the RGB fog image to be processed according to the image characteristics; training the defogging model by adopting the sample data set to obtain a trained defogging model; and defogging the real photographed fog image through the trained defogging model. According to the invention, the red channel guide module is used for guiding the main body module to carry out image defogging, so that more image detail information can be reserved, and the defogging effect of the invention is better.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image defogging method and system based on red channel guidance.
Background
Due to the low visibility in the foggy weather, the pictures shot in the foggy weather are also unclear, and in order to obtain clear pictures in the foggy weather, the pictures shot in the foggy weather need to be subjected to defogging treatment.
Currently, image defogging is performed by optimization algorithms or convolutional neural networks based on traditional atmospheric scattering models. However, the fog generated based on the traditional atmospheric scattering model has a certain difference from the real fog, so that the convolution neural network trained by the data set synthesized by the traditional atmospheric scattering model has poor defogging effect on the real photographed fog image.
Disclosure of Invention
In view of the above, an image defogging method and system based on red channel guidance are provided to solve the problem of poor defogging effect in the related art.
The invention adopts the following technical scheme:
in a first aspect, the present invention provides an image defogging method based on red channel guidance, including:
constructing a sample data set;
constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics;
training the defogging model by adopting the sample data set to obtain a trained defogging model;
and defogging the real photographed fog image through the trained defogging model.
Preferably, the red channel guidance module includes 1 first input layer, 1 scale-conversion convolutional layer, 4 void convolutional layers, 2 convolutional layers, 1 connection layer, and 1 normalization layer;
the connection mode of each layer is as follows: the first input layer → the scale-change buildup layer → the cavity buildup layer 1 → the cavity buildup layer 2 → the cavity buildup layer 3 → the cavity buildup layer 4 → the connection layer → the buildup layer 1 → the buildup layer 2 → the normalization layer;
after the first input layer acquires the R channel graph, extracting a first sub-feature graph in the R channel graph, and sending the first sub-feature graph to the scale conversion convolutional layer;
the scale conversion convolutional layer performs scale conversion processing on the first sub-feature graph to obtain first sub-feature graphs under different scales, and sends the first sub-feature graphs under different scales to the cavity convolutional layer;
the cavity convolution layer obtains second sub-feature maps under different perception fields according to the first sub-feature maps under different scales; the second sub-feature diagram is used for supporting the main body module to carry out defogging operation on the RGB fog diagram to be processed.
Preferably, the scale-conversion convolutional layer comprises an up-sampling layer and a down-sampling layer;
the up-sampling layer is used for amplifying the scale of the first sub-feature map;
the down-sampling layer is used for reducing the scale of the first sub-feature map.
Preferably, the body module includes: the device comprises a sub-coding module, a sub-decoding module and a characteristic reconstruction module;
the characteristic reconstruction module is respectively connected with the sub-coding module and the sub-decoding module;
the sub-coding module is used for extracting image features in the RGB fog image to be processed and sending the extracted image features in the RGB fog image to be processed to the feature reconstruction module;
the characteristic reconstruction module is used for integrating the image characteristics in the RGB fog image to be processed output by the sub-coding module and sending the integrated image characteristics in the RGB fog image to be processed to the sub-decoding module;
the decoding module is used for carrying out image reconstruction on the image characteristics in the integrated RGB fog map to be processed to obtain a target defogging map;
preferably, the sub-coding module includes: the system comprises a first guide application unit, a first multi-scale dense fusion unit and an encoder;
the connection mode of each component part is as follows: second input layer → first guiding application unit → first multi-scale dense fusion unit → encoder;
the second input layer is used for extracting a third sub-feature map of the RGB fog map to be processed after the RGB fog map to be processed is obtained, and sending the third sub-feature map to the first guide application unit;
the first guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-coding module, so that the sub-coding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the first multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and sending the fused feature map to the encoder;
the encoder is used for transforming the scale of the fused feature map and extracting the features in the fused feature map.
Preferably, the sub-decoding module includes: the second guiding application unit, the second multi-scale dense fusion unit, the decoder and the jump connection feature fusion unit;
the connection mode of each component part is as follows: the third input layer → the second multi-scale dense fusion unit → the skip-join feature fusion unit → the second guidance application unit → the decoder;
the third input layer is used for acquiring image characteristics in the RGB fog image to be processed sent by the encoder;
the second multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;
the skip-connection feature fusion unit is used for receiving a target feature map which is sent by an encoder of the sub-encoding module and has the same scale size as a current feature map, and fusing the target feature map and the current feature map;
the second guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the decoder is used for transforming the scale of the received features and reconstructing the image of the features after the scale is transformed to obtain the target defogging image.
Preferably, the sample data set includes: a training data set;
the training data set comprises a plurality of pairs of image pairs; each pair of the images consists of two corresponding RGB fog images and a clear image;
training the defogging model by adopting the sample data set to obtain the trained defogging model, wherein the training comprises the following steps:
the defogging model acquires a target RGB fog image;
the defogging model processes the RGB fog map to obtain a defogging map;
calculating a loss function of the defogging map according to the defogging map and a clearness map corresponding to the target RGB fogging map;
updating model parameters of the defogging model according to the loss function;
and repeatedly executing the steps until the loss function meets the requirement of a preset loss value.
Preferably, the sample data set further comprises: testing the data set;
after the defogging model is trained by adopting the sample data set and the trained defogging model is obtained, the method further comprises the following steps:
and testing the trained defogging model by adopting the test data set to obtain a test result.
Preferably, the test result comprises a peak signal-to-noise ratio.
In a second aspect, the present invention further provides an image defogging system based on red channel guidance, which is applied to the method in the first aspect of the present invention, and the system includes:
the sample data construction module is used for constructing a sample data set;
the module construction module is used for constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics;
the model training module is used for training the defogging model by adopting the sample data set to obtain the trained defogging model;
and the defogging module is used for defogging the real photographed fog image through the trained defogging model.
By adopting the technical scheme, the image defogging method based on the red channel guidance comprises the following steps: constructing a sample data set; constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics; training the defogging model by adopting the sample data set to obtain a trained defogging model; and defogging the real photographed fog image through the trained defogging model. Based on this, because the transmissivity of the R channel is the highest among the R channel, the G channel and the B channel, light can penetrate through the R channel most easily, and compared with the other two channels, the R channel has more picture detail information.
It should be noted that R, G and B represent the colors of the red, green and blue channels, respectively. An RGB image is an image displayed in an RGB color mode, and such an image can be viewed only through a medium for expressing colors by using light, such as a television or a computer.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of an image defogging method based on red channel guidance according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an image defogging system based on red channel guidance according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without any inventive step, are within the scope of the present invention.
Fig. 1 is a schematic flowchart of an image defogging method based on red channel guidance according to an embodiment of the present invention. As shown in fig. 1, the image defogging method based on red channel guidance of the embodiment includes:
s101, constructing a sample data set.
Specifically, the sample data set comprises a training data set, a fog image and a clear image which correspond to each other one by one, and the fog image and the clear image which correspond to each other are pictures in the same shooting scene. The fog map is generated by adding synthetic fog based on an atmospheric scattering model to a clear map corresponding thereto.
S102, constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog image to be processed according to the image characteristics.
In detail, it is known that, as the wavelength of the light is longer, the reduction of the light by the fog is weaker, and therefore, in the RGB three channels, the reduction corresponding to the red channel with the longest wavelength is least, that is, the fog on the red channel map is lighter than the fog on the other two channel maps, and therefore, the red channel map has more image details than the other two channel maps, so that the red channel guide module is used in the present embodiment to guide the main body module to perform the defogging, so that the present embodiment can obtain a better defogging effect.
More specifically, the red channel guidance module is configured to obtain an R channel map of an RGB fog map to be processed, extract an image feature of the R channel map, and send the image feature to the main module, and the main module obtains picture detail information of the RGB fog map through the R channel map and performs defogging on the RGB fog map according to the detail information, so that the RGB fog map can retain the picture detail information of the RGB fog map obtained through the R channel map, and the defogging effect of the embodiment is good.
S103, training the defogging model by adopting the sample data set to obtain the trained defogging model.
Specifically, first, training parameters are set, and then the defogging model is trained using python and the deep learning framework pytorch to converge, thereby obtaining a target defogging model (a defogging model after training).
In more detail, first, the training learning rate is set to 0.00005, and is reduced to 0.9 times after each 10 iterations, and the total number of iterations is set to 100. And then, inputting the defogging image in the training data set into the defogging model, calculating the loss of the image output by the defogging model and the corresponding clear image, and defining the current defogging model as a target defogging model when the calculation result is converged.
And S104, defogging the real photographed fog image through the trained defogging model.
Specifically, after the real-shot fog image is input into the trained defogging model, the trained defogging model performs defogging operation on the real-shot fog image and outputs a defogged clear image.
In summary, in the present embodiment, with the above technical solutions, an image defogging method based on red channel guidance includes: constructing a sample data set; constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics; training the defogging model by adopting the sample data set to obtain a trained defogging model; and defogging the real photographed fog image through the trained defogging model. Based on this, because among R passageway, G passageway and the B passageway, the transmissivity of R passageway is the highest, consequently, the light passes through the R passageway most easily, compares in other two passageways, has more picture detail information on the R passageway, and this embodiment guides the main part module through red passageway guide module and carries out the image defogging, can keep more picture detail information for the defogging effect of this embodiment is better.
Preferably, the red channel guidance module includes 1 first input layer, 1 scale-conversion convolutional layer, 4 void convolutional layers, 2 convolutional layers, 1 connection layer, and 1 normalization layer;
the connection mode of each layer is as follows: the first input layer → the scale-change buildup layer → the cavity buildup layer 1 → the cavity buildup layer 2 → the cavity buildup layer 3 → the cavity buildup layer 4 → the connection layer → the buildup layer 1 → the buildup layer 2 → the normalization layer;
after the first input layer acquires the R channel graph, extracting a first sub-feature graph in the R channel graph, and sending the first sub-feature graph to the scale conversion convolutional layer;
the scale conversion convolutional layer performs scale conversion processing on the first sub-feature graph to obtain first sub-feature graphs under different scales, and sends the first sub-feature graphs under different scales to the cavity convolutional layer;
the cavity convolution layer obtains second sub-feature maps under different perception fields according to the first sub-feature maps under different scales; the second sub-feature diagram is used for supporting the main body module to carry out defogging operation on the RGB fog diagram to be processed.
Specifically, the scale-conversion convolutional layer comprises an up-sampling layer and a down-sampling layer; the up-sampling layer is used for amplifying the scale of the first sub-feature map; the down-sampling layer is used for reducing the scale of the first sub-feature map.
The construction method of the red channel guide module is as follows:
1. the layers are connected as follows:
the first input layer → the scale-conversion buildup layer (upsampling layer) → the cavity buildup layer 1 → the cavity buildup layer 2 → the cavity buildup layer 3 → the cavity buildup layer 4 → the connecting layer → the buildup layer 1 → the buildup layer 2 → the normalization layer;
2. step 1 is repeatedly executed three times;
3. replacing the up-sampling layer with the down-sampling layer;
4. and repeating the step 1 for three times to obtain the red channel guide module.
The first input layer receives a red channel in an input image, and shallow layer features are obtained preliminarily. And then carrying out scale transformation to obtain feature maps with different scales. Then, deep characteristic maps of different receptive fields are extracted through 4 void convolution layers, and the deep characteristic maps have more detailed information and provide supplement and guidance for subsequent defogging.
The red channel guiding module extracts the detail information missing from other channels from the red channel R and sends the information to the corresponding position of the main body module through linear mapping so as to guide the main body module to carry out defogging. The scale-transform convolutional layer includes an up-sampling layer and a down-sampling layer, and can make the scale (i.e. size) of the feature map in the red channel guide module the same as the scale of the feature map in the current body module.
Wherein the linear mapping is: after the output of the red channel guide module enters the main module, two parallel weight parameters are calculated through two parallel convolutions (ReLU layers), and the two parameters are applied to carry out linear mapping on the characteristic diagram of the main module.
Preferably, the body module includes: the device comprises a sub-coding module, a sub-decoding module and a characteristic reconstruction module;
the characteristic reconstruction module is respectively connected with the sub-coding module and the sub-decoding module;
the sub-coding module is used for extracting image features in the RGB fog image to be processed and sending the extracted image features in the RGB fog image to be processed to the feature reconstruction module;
the characteristic reconstruction module is used for integrating the image characteristics in the RGB fog image to be processed output by the sub-coding module and sending the integrated image characteristics in the RGB fog image to be processed to the sub-decoding module;
the decoding module is used for carrying out image reconstruction on the image characteristics in the integrated RGB fog map to be processed to obtain a target defogging map;
preferably, the sub-coding module includes: the system comprises a first guide application unit, a first multi-scale dense fusion unit and an encoder;
the connection mode of each component part is as follows: second input layer → first guiding application unit → first multi-scale dense fusion unit → encoder;
the second input layer is used for extracting a third sub-feature map of the RGB fog map to be processed after the RGB fog map to be processed is obtained, and sending the third sub-feature map to the first guide application unit;
the first guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-coding module, so that the sub-coding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the first multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and sending the fused feature map to the encoder;
the encoder is used for transforming the scale of the fused feature map and extracting the features in the fused feature map.
Specifically, the construction method of the sub-coding module is as follows:
1. the layers are connected as follows:
second input layer → first guiding application unit → first multi-scale dense fusion unit → encoder;
2. step 1 was performed repeatedly three times.
The input of the second input layer is a three-channel foggy picture, and the second input layer extracts a shallow feature map of the three-channel foggy picture for extracting and processing subsequent deep features.
More specifically, the first guidance application unit includes: the two first sub-convolution layers and the linear computation subunit are connected in a mode that: first sub convolution layer 1 → first sub convolution layer 2 → linear computation subunit; the linear calculation formula adopted by the linear calculation subunit is as follows:
Il＝conv1(Ir)×I0+conv2(Ir)
wherein, conv1Representing the first sub-convolution layer 1, conv2Denotes a first sub-convolution layer 2, I0Representing the input of the first sub-convolutional layer 1.
The first multi-scale dense fusion unit includes: a first sub-input layer, a number of first sub-connection layers, a number of first up-sampled convolutional layers and a number of first down-sampled convolutional layers, the number of each component depending on the scale of the current input. Taking the third dimension as an example, the connection mode of the components is as follows: first sub-input layer → first up-sampling convolution layer 1 → first up-sampling convolution layer 2 → first up-sampling convolution layer 3 → first sub-connection layer 1 → first down-sampling convolution layer 2 → first down-sampling convolution layer 3 → first sub-connection layer 2 → first up-sampling convolution layer 4 → first up-sampling convolution layer 5 → first sub-connection layer 3 → first down-sampling convolution layer 4 → first down-sampling convolution layer 5 → first sub-connection layer 4 → first up-sampling convolution layer 6 → first sub-connection layer 5 → first down-sampling convolution layer 6. The first sub-input layer inputs the output feature maps of the first three scales into the part of the neural network, and the feature maps of the 1 st and 2 nd scales are fused with the feature map of the current 3 rd scale through the different numbers of up/first down-sampling convolutional layers.
The encoder includes: a second sub-input layer, 1 second downsampled convolutional layer, 11 second sub-convolutional layers, 6 first active layers, 7 second sub-connection layers, 2 first average pooling layers, and 1 maximum pooling layer. The connection mode of each component part is as follows: the second sub input layer → the second down-sampled convolution layer → the second sub-convolution layer 1 → the first active layer 1 → the second sub-convolution layer 2 → the second sub-link layer 1 → the second sub-convolution layer 3 → the first active layer 2 → the second sub-convolution layer 4 → the second sub-link layer 2 → the second sub-convolution layer 5 → the first active layer 3 → the second sub-convolution layer 6 → the second sub-link layer 3 → the second sub-convolution layer 7 → the first active layer 4 → the second sub-convolution layer 8 → the second sub-link layer 4 → the maximum value pooling layer → the first average value pooling layer 1 → the second sub-link layer 5 → the second sub-convolution layer 9 → the first active layer 5 → the second sub-link layer 6 → the first average value pooling layer 2 → the second sub-convolution layer 10 → the first active layer 6 → the second sub-convolution layer 11 → the first active layer 7 → the second sub-link layer 7.
Preferably, the sub-decoding module includes: the second guiding application unit, the second multi-scale dense fusion unit, the decoder and the jump connection feature fusion unit;
the connection mode of each component part is as follows: the third input layer → the second multi-scale dense fusion unit → the skip-join feature fusion unit → the second guidance application unit → the decoder;
the third input layer is used for acquiring image characteristics in the RGB fog image to be processed sent by the encoder;
the second multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;
the skip-connection feature fusion unit is used for receiving a target feature map which is sent by an encoder of the sub-encoding module and has the same scale size as a current feature map, and fusing the target feature map and the current feature map;
the second guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the decoder is used for transforming the scale of the received features and reconstructing the image of the features after the scale is transformed to obtain the target defogging image.
In detail, the second guiding application unit and the second multi-scale dense fusion unit have the same structure as the sub-coding module, respectively, and the difference is only that the up/down sampling convolution layer is exchanged, which is not described herein again. The decoder structure is the same as the encoder structure, and the difference is only to replace the downsampled convolutional layer with an upsampled convolutional layer, which is not described herein again. The connection mode of each component part is as follows: the third input layer → the second multi-scale dense fusion unit → the skip-join feature fusion unit → the second guidance application unit → the decoder; and repeated four times.
The jump connection feature fusion unit comprises: a third sub-input layer, 2 third sub-connection layers, 2 third sub-convolution layers, 1 second average pooling layer, and 1 second active layer. The connection mode of each component part is as follows: the third sub input layer- > the third sub connecting layer 1- > the second average pooling layer- > the third sub convolution layer 1- > the third sub convolution layer 2- > the second average pooling layer- > the second active layer- > the third sub connecting layer 2. The input of the second sub-input layer is the characteristic diagram of the current scale and the characteristic diagram of the corresponding scale of the coding part, so as to obtain more detailed information in the characteristic diagram output by the coder and fully utilize the characteristic diagram output by the coder.
Preferably, the sample data set includes: a training data set;
the training data set comprises a plurality of pairs of image pairs; each pair of the images consists of two corresponding RGB fog images and a clear image;
training the defogging model by adopting the sample data set to obtain the trained defogging model, wherein the training comprises the following steps:
the defogging model acquires a target RGB fog image;
the defogging model processes the RGB fog map to obtain a defogging map;
calculating a loss function of the defogging map according to the defogging map and a clearness map corresponding to the target RGB fogging map;
updating model parameters of the defogging model according to the loss function;
and repeatedly executing the steps until the loss function meets the requirement of a preset loss value.
In detail, a target RGB fog map is input into a main body module of a constructed defogging model, an R channel map of the target RGB fog map is input into a red channel guide module of the defogging model, the main body module and the red channel guide module are fused for defogging, the defogging module outputs the defogging map, a loss function of the defogging map is calculated by referring to a clearness map corresponding to the target RGB fog map, and finally, model parameters of the defogging model are updated according to the loss function. Wherein the loss function is as follows:
wherein, J1(x) An output diagram after the network defogging is shown; j. the design is a square2(x) A clear graph in the training data is shown.
Wherein the Adam optimizer is adopted to update the model parameters of the defogging model.
Preferably, the sample data set further comprises: testing the data set;
after the defogging model is trained by adopting the sample data set and the trained defogging model is obtained, the method further comprises the following steps:
and testing the trained defogging model by adopting the test data set to obtain a test result.
Specifically, after model training is completed, the defogging image in the test data set is input into the trained defogging model to obtain an output defogging image, and the index is evaluated by calculating parameters such as the peak signal-to-noise ratio of the defogging image to evaluate the network defogging effect.
Based on a general inventive concept, the present invention further provides an image defogging system based on red channel guidance, which is applied to the image defogging method based on red channel guidance according to the above embodiment of the present invention.
Fig. 2 is a schematic structural diagram of an image defogging system based on red channel guidance according to an embodiment of the present invention. As shown in fig. 2, the image defogging system based on the red channel guidance of the embodiment includes: the system comprises a sample data construction module 21, a module construction module 22, a model training module 23 and a defogging module 24.
The sample data construction module 21 is configured to construct a sample data set; a module construction module 22 for constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics; the model training module 23 is configured to train the defogging model by using the sample data set to obtain a trained defogging model; and the defogging module 24 is used for defogging the real photographed fog image through the trained defogging model.
Preferably, the red channel guidance module includes 1 first input layer, 1 scale-conversion convolutional layer, 4 void convolutional layers, 2 convolutional layers, 1 connection layer, and 1 normalization layer;
the connection mode of each layer is as follows: the first input layer → the scale-change buildup layer → the cavity buildup layer 1 → the cavity buildup layer 2 → the cavity buildup layer 3 → the cavity buildup layer 4 → the connection layer → the buildup layer 1 → the buildup layer 2 → the normalization layer;
after the first input layer acquires the R channel graph, extracting a first sub-feature graph in the R channel graph, and sending the first sub-feature graph to the scale conversion convolutional layer;
the scale conversion convolutional layer performs scale conversion processing on the first sub-feature graph to obtain first sub-feature graphs under different scales, and sends the first sub-feature graphs under different scales to the cavity convolutional layer;
the cavity convolution layer obtains second sub-feature maps under different perception fields according to the first sub-feature maps under different scales; the second sub-feature diagram is used for supporting the main body module to carry out defogging operation on the RGB fog diagram to be processed.
Preferably, the scale-conversion convolutional layer comprises an up-sampling layer and a down-sampling layer;
the up-sampling layer is used for amplifying the scale of the first sub-feature map;
the down-sampling layer is used for reducing the scale of the first sub-feature map.
Preferably, the body module includes: the device comprises a sub-coding module, a sub-decoding module and a characteristic reconstruction module;
the characteristic reconstruction module is respectively connected with the sub-coding module and the sub-decoding module;
the sub-coding module is used for extracting image features in the RGB fog image to be processed and sending the extracted image features in the RGB fog image to be processed to the feature reconstruction module;
the characteristic reconstruction module is used for integrating the image characteristics in the RGB fog image to be processed output by the sub-coding module and sending the integrated image characteristics in the RGB fog image to be processed to the sub-decoding module;
the decoding module is used for carrying out image reconstruction on the image characteristics in the integrated RGB fog map to be processed to obtain a target defogging map;
preferably, the sub-coding module includes: the system comprises a first guide application unit, a first multi-scale dense fusion unit and an encoder;
the connection mode of each component part is as follows: second input layer → first guiding application unit → first multi-scale dense fusion unit → encoder;
the second input layer is used for extracting a third sub-feature map of the RGB fog map to be processed after the RGB fog map to be processed is obtained, and sending the third sub-feature map to the first guide application unit;
the first guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-coding module, so that the sub-coding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the first multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and sending the fused feature map to the encoder;
the encoder is used for transforming the scale of the fused feature map and extracting the features in the fused feature map.
Preferably, the sub-decoding module includes: the second guiding application unit, the second multi-scale dense fusion unit, the decoder and the jump connection feature fusion unit;
the connection mode of each component part is as follows: the third input layer → the second multi-scale dense fusion unit → the skip-join feature fusion unit → the second guidance application unit → the decoder;
the third input layer is used for acquiring image characteristics in the RGB fog image to be processed sent by the encoder;
the second multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;
the skip-connection feature fusion unit is used for receiving a target feature map which is sent by an encoder of the sub-encoding module and has the same scale size as a current feature map, and fusing the target feature map and the current feature map;
the second guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the decoder is used for transforming the scale of the received features and reconstructing the image of the features after the scale is transformed to obtain the target defogging image.
Preferably, the sample data set includes: a training data set;
the training data set comprises a plurality of pairs of image pairs; each pair of the images consists of two corresponding RGB fog images and a clear image;
the model training module 23 is specifically configured to implement the following method:
the defogging model acquires a target RGB fog image;
the defogging model processes the RGB fog map to obtain a defogging map;
calculating a loss function of the defogging map according to the defogging map and a clearness map corresponding to the target RGB fogging map;
updating model parameters of the defogging model according to the loss function;
and repeatedly executing the steps until the loss function meets the requirement of a preset loss value.
Preferably, the sample data set further comprises: testing the data set;
the image defogging system based on the red channel guidance of the embodiment further comprises: and the test module is used for testing the trained defogging model by adopting the test data set to obtain a test result.
Preferably, the test result comprises a peak signal-to-noise ratio.
It should be noted that the image defogging system based on the red channel guidance in the present embodiment and the image defogging method based on the red channel guidance in the foregoing embodiments are based on a general inventive concept, and have the same or corresponding execution processes and beneficial effects, and are not described herein again.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that the terms "first," "second," and the like in the description of the present invention are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present invention, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow diagrams or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present invention includes additional implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. An image defogging method based on red channel guidance is characterized by comprising the following steps:
constructing a sample data set;
constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics;
training the defogging model by adopting the sample data set to obtain a trained defogging model;
and defogging the real photographed fog image through the trained defogging model.
2. The red channel guidance-based image defogging method according to claim 1, wherein the red channel guidance module comprises 1 first input layer, 1 scale conversion convolutional layer, 4 void convolutional layers, 2 convolutional layers, 1 connection layer and 1 normalization layer;
the connection mode of each layer is as follows: the first input layer → the scale-change buildup layer → the cavity buildup layer 1 → the cavity buildup layer 2 → the cavity buildup layer 3 → the cavity buildup layer 4 → the connection layer → the buildup layer 1 → the buildup layer 2 → the normalization layer;
after the first input layer acquires the R channel graph, extracting a first sub-feature graph in the R channel graph, and sending the first sub-feature graph to the scale conversion convolutional layer;
the scale conversion convolutional layer performs scale conversion processing on the first sub-feature graph to obtain first sub-feature graphs under different scales, and sends the first sub-feature graphs under different scales to the cavity convolutional layer;
the cavity convolution layer obtains second sub-feature maps under different perception fields according to the first sub-feature maps under different scales; the second sub-feature diagram is used for supporting the main body module to carry out defogging operation on the RGB fog diagram to be processed.
3. The red channel-guided image defogging method according to claim 2, wherein said scale conversion convolutional layer comprises an up-sampling layer and a down-sampling layer;
the up-sampling layer is used for amplifying the scale of the first sub-feature map;
the down-sampling layer is used for reducing the scale of the first sub-feature map.
4. The red channel guidance-based image defogging method according to claim 1, wherein the main body module comprises: the device comprises a sub-coding module, a sub-decoding module and a characteristic reconstruction module;
the characteristic reconstruction module is respectively connected with the sub-coding module and the sub-decoding module;
the sub-coding module is used for extracting image features in the RGB fog image to be processed and sending the extracted image features in the RGB fog image to be processed to the feature reconstruction module;
the characteristic reconstruction module is used for integrating the image characteristics in the RGB fog image to be processed output by the sub-coding module and sending the integrated image characteristics in the RGB fog image to be processed to the sub-decoding module;
and the decoding module is used for carrying out image reconstruction on the image characteristics in the integrated RGB fog map to be processed to obtain a target defogging map.
5. The red channel guidance-based image defogging method according to claim 4, wherein the sub-encoding module comprises: the system comprises a first guide application unit, a first multi-scale dense fusion unit and an encoder;
the connection mode of each component part is as follows: second input layer → first guiding application unit → first multi-scale dense fusion unit → encoder;
the second input layer is used for extracting a third sub-feature map of the RGB fog map to be processed after the RGB fog map to be processed is obtained, and sending the third sub-feature map to the first guide application unit;
the first guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-coding module, so that the sub-coding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the first multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and sending the fused feature map to the encoder;
the encoder is used for transforming the scale of the fused feature map and extracting the features in the fused feature map.
6. The red channel guidance-based image defogging method according to claim 4, wherein said sub-decoding module comprises: the second guiding application unit, the second multi-scale dense fusion unit, the decoder and the jump connection feature fusion unit;
the connection mode of each component part is as follows: the third input layer → the second multi-scale dense fusion unit → the skip-join feature fusion unit → the second guidance application unit → the decoder;
the third input layer is used for acquiring image characteristics in the RGB fog image to be processed sent by the encoder;
the second multi-scale dense fusion unit is used for fusing the received feature map sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;
the skip-connection feature fusion unit is used for receiving a target feature map which is sent by an encoder of the sub-encoding module and has the same scale size as a current feature map, and fusing the target feature map and the current feature map;
the second guide application unit is used for acquiring the image characteristics of the R channel map and sending the acquired image characteristics of the R channel map to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog map to be processed according to the image characteristics of the R channel map;
the decoder is used for transforming the scale of the received features and reconstructing the image of the features after the scale is transformed to obtain the target defogging image.
7. The red channel guidance-based image defogging method according to claim 1, wherein said sample data set comprises: a training data set;
the training data set comprises a plurality of pairs of image pairs; each pair of the images consists of two corresponding RGB fog images and a clear image;
training the defogging model by adopting the sample data set to obtain the trained defogging model, wherein the training comprises the following steps:
the defogging model acquires a target RGB fog image;
the defogging model processes the RGB fog map to obtain a defogging map;
calculating a loss function of the defogging map according to the defogging map and a clearness map corresponding to the target RGB fogging map;
updating model parameters of the defogging model according to the loss function;
and repeatedly executing the steps until the loss function meets the requirement of a preset loss value.
8. The red channel guidance-based image defogging method according to claim 7, wherein said sample data set further comprises: testing the data set;
after the defogging model is trained by adopting the sample data set and the trained defogging model is obtained, the method further comprises the following steps:
and testing the trained defogging model by adopting the test data set to obtain a test result.
9. The red channel guidance-based image defogging method according to claim 8, wherein said test results comprise a peak signal-to-noise ratio.
10. The image defogging system based on the red channel guidance is applied to the image defogging method based on the red channel guidance according to claim 1, and is characterized by comprising the following steps:
the sample data construction module is used for constructing a sample data set;
the module construction module is used for constructing a defogging model; the defogging model comprises a red channel guide module and a main body module; the red channel guide module is used for acquiring an R channel image of an RGB fog image to be processed, extracting image features of the R channel image and sending the image features to the main module; the main body module is used for carrying out defogging operation on the RGB fog map to be processed according to the image characteristics;
the model training module is used for training the defogging model by adopting the sample data set to obtain the trained defogging model;
and the defogging module is used for defogging the real photographed fog image through the trained defogging model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111394646.9A CN114119403A (en) | 2021-11-23 | 2021-11-23 | Image defogging method and system based on red channel guidance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111394646.9A CN114119403A (en) | 2021-11-23 | 2021-11-23 | Image defogging method and system based on red channel guidance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114119403A true CN114119403A (en) | 2022-03-01 |
Family
ID=80440270
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111394646.9A Pending CN114119403A (en) | 2021-11-23 | 2021-11-23 | Image defogging method and system based on red channel guidance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114119403A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863242A (en) * | 2022-04-26 | 2022-08-05 | 北京拙河科技有限公司 | Deep learning network optimization method and system for image recognition |
-
2021
- 2021-11-23 CN CN202111394646.9A patent/CN114119403A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863242A (en) * | 2022-04-26 | 2022-08-05 | 北京拙河科技有限公司 | Deep learning network optimization method and system for image recognition |
| CN114863242B (en) * | 2022-04-26 | 2022-11-29 | 北京拙河科技有限公司 | Deep learning network optimization method and system for image recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113362223B (en) | Image super-resolution reconstruction method based on attention mechanism and two-channel network | |
| CN109472270A (en) | Image style conversion method, device and equipment | |
| CN111915619A (en) | Full convolution network semantic segmentation method for dual-feature extraction and fusion | |
| CN113012172A (en) | AS-UNet-based medical image segmentation method and system | |
| CN110689599A (en) | 3D visual saliency prediction method for generating countermeasure network based on non-local enhancement | |
| CN110175986A (en) | A kind of stereo-picture vision significance detection method based on convolutional neural networks | |
| CN109003297A (en) | A kind of monocular depth estimation method, device, terminal and storage medium | |
| CN114283158A (en) | Retinal blood vessel image segmentation method and device and computer equipment | |
| CN110276731A (en) | Endoscopic image color restoring method and device | |
| CN116343144B (en) | Real-time target detection method integrating visual perception and self-adaptive defogging | |
| CN113762264A (en) | Multi-encoder fused multispectral image semantic segmentation method | |
| CN113744146A (en) | Image defogging method based on contrast learning and knowledge distillation | |
| CN114359297A (en) | Attention pyramid-based multi-resolution semantic segmentation method and device | |
| CN115330620A (en) | Image defogging method based on cyclic generation countermeasure network | |
| CN112509106A (en) | Document picture flattening method, device and equipment | |
| CN116569218A (en) | Image processing method and image processing apparatus | |
| CN110942484A (en) | Camera self-motion estimation method based on occlusion perception and feature pyramid matching | |
| CN114565539B (en) | Image defogging method based on online knowledge distillation | |
| CN114119403A (en) | Image defogging method and system based on red channel guidance | |
| CN117079105A (en) | Remote sensing image spatial spectrum fusion method and device, electronic equipment and storage medium | |
| CN113298744B (en) | End-to-end infrared and visible light image fusion method | |
| CN113379606B (en) | Face super-resolution method based on pre-training generation model | |
| CN113112441B (en) | Multi-band low-resolution image synchronous fusion method based on dense network and local brightness traversal operator | |
| CN113128517A (en) | Tone mapping image mixed visual feature extraction model establishment and quality evaluation method | |
| CN117408924A (en) | Low-light image enhancement method based on multiple semantic feature fusion network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |