CN113222847A - Image defogging method based on generation countermeasure network - Google Patents
Image defogging method based on generation countermeasure network Download PDFInfo
- Publication number
- CN113222847A CN113222847A CN202110542673.XA CN202110542673A CN113222847A CN 113222847 A CN113222847 A CN 113222847A CN 202110542673 A CN202110542673 A CN 202110542673A CN 113222847 A CN113222847 A CN 113222847A
- Authority
- CN
- China
- Prior art keywords
- image
- defogging
- network
- countermeasure network
- fog
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000000694 effects Effects 0.000 claims abstract description 12
- 238000011156 evaluation Methods 0.000 claims abstract description 8
- 238000003384 imaging method Methods 0.000 claims abstract description 8
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 7
- 238000005457 optimization Methods 0.000 claims abstract description 6
- 238000011158 quantitative evaluation Methods 0.000 claims abstract description 5
- 238000002474 experimental method Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 12
- 238000011160 research Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 230000015556 catabolic process Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000006731 degradation reaction Methods 0.000 claims description 4
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 3
- 230000007547 defect Effects 0.000 claims description 3
- 238000009795 derivation Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 238000002834 transmittance Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an image defogging method based on a generation countermeasure network, and relates to the technical field of image defogging; the method comprises the following steps: step one, researching defogging of a dark channel, and researching defogging from a background model of fog image imaging; step two, learning a convolutional neural network to carry out image defogging; step three, improving a cycleGAN network; verifying the defogging effect of the defogged clear image through an experiment, performing qualitative and quantitative evaluation, and analyzing a comparison result to compare whether indexes of the improved algorithm based on the generation countermeasure network and the dark channel first-checking algorithm in the aspects of subjective evaluation and objective evaluation are improved or not; the invention can realize defogging of the image, can ensure the definition of the image, and has convenient use and high stability; further optimization of the defogging effect of the image is achieved using generation of a countermeasure network for the characteristics of foggy weather.
Description
Technical Field
The invention belongs to the technical field of image defogging, and particularly relates to an image defogging method based on a generation countermeasure network.
Background
The 21 st century is an information and intelligent era, and images as the most common and effective information carriers for bearing information play a very important role in the production and life of people. With the continuous development of science and technology, the popularization of products such as mobile phones and the like is more and more extensive, people can not leave digital images, the digital images are widely applied in scientific research or social production, and a large number of digital images are acquired and analyzed and then are applied to various scientific research and production practices, and the digital images are widely applied to occasions such as target detection, outdoor photography and the like. However, the quality of the obtained digital image is greatly reduced due to the existence of the fog, and if the digital image is not processed in a targeted manner, the data may not meet the requirements of corresponding research and application. The degradation of image quality in foggy weather not only reduces the ornamental value of the foggy weather, but also seriously influences the effective implementation of various computer vision tasks taking image videos as main processing objects. For example, the current monitoring system is widely applied to scenes such as public places, road traffic and the like, and if an image obtained by the monitoring system is fuzzy in haze weather, the subsequent work such as vehicle detection, face recognition and the like can be directly influenced; on the other hand, the aerial photography system of present unmanned aerial vehicle also by extensive be applied to in aspects such as geographical survey and drawing, disaster detection, the haze image can serious interference light sense imaging system, this has just led to unmanned aerial vehicle's the decline of the image quality of taking photo by plane, has brought adverse effect to the normal clear of relevant work, has certain potential safety hazard even. The image processing technology is developed rapidly in recent years, the processed image has higher precision, the embodied image information is more comprehensive, the observed object information is more visual, and the image processing technology can be better used for scientific research and application. Therefore, the research has very important practical significance and application significance, and is a hot problem in the current image processing field.
Disclosure of Invention
To solve the problems of the background art; the invention aims to provide an image defogging method based on a generation countermeasure network.
The invention relates to an image defogging method based on a generation countermeasure network, which comprises the following steps:
step one, researching defogging of a dark channel, researching defogging from a background model of fog image imaging, and gradually deriving an estimated value of transmissivity in a derivation model on the basis of an atmospheric scattering model, so that the basic process of fog image imaging and the specific meanings of each parameter in the model are deeply understood, a theoretical cushion is laid for later-stage research, and problems and defects occurring in the algorithm are used as important breakthrough openings of follow-up research;
step two, learning a convolutional neural network to carry out image defogging, estimating atmospheric degradation transmittance by using a DehazeNet model established by a CNN deep structure, and using a bilateral correction linear unit as an activation function to improve the recovery quality of an image;
step three, improving a CycleGAN network, building a neural network and training to complete the conversion from a fog image to a clear image, finally realizing the optimization of the defogging effect, and judging the continuous optimization of various losses through the generation loss, so that the quality of the generated image is continuously improved;
and step four, verifying the defogging effect of the defogged clear image through an experiment, performing qualitative and quantitative evaluation, analyzing a comparison result, and comparing whether the indexes of the improved algorithm based on the generation countermeasure network and the dark channel first-checking algorithm in the aspects of subjective evaluation and objective evaluation are improved or not.
Compared with the prior art, the invention has the beneficial effects that:
the defogging of the image can be realized, the definition of the image can be ensured, the use is convenient, and the stability is high;
and secondly, aiming at the characteristics of foggy days, further optimizing the defogging effect of the image by using the generation countermeasure network.
Detailed Description
The specific implementation mode adopts the following technical scheme: the method comprises the following steps:
step one, researching defogging of a dark channel, researching defogging from a background model of fog image imaging, and gradually deriving an estimated value of transmissivity in a derivation model on the basis of an atmospheric scattering model, so that the basic process of fog image imaging and the specific meanings of each parameter in the model are deeply understood, a theoretical cushion is laid for later-stage research, and problems and defects occurring in the algorithm are used as important breakthrough openings of follow-up research;
step two, learning a convolutional neural network to carry out image defogging, estimating atmospheric degradation transmittance by using a DehazeNet model established by a CNN deep structure, and using a bilateral correction linear unit as an activation function to improve the recovery quality of an image;
step three, improving a CycleGAN network, building a neural network and training to complete the conversion from a fog image to a clear image, finally realizing the optimization of the defogging effect, and continuously improving the quality of the generated image by continuously optimizing various losses such as the generation loss judgment loss and the like;
and step four, verifying the defogging effect of the defogged clear image through an experiment, performing qualitative and quantitative evaluation, analyzing a comparison result, and comparing whether the indexes of the improved algorithm based on the generation countermeasure network and the dark channel first-checking algorithm in the aspects of subjective evaluation and objective evaluation are improved or not.
The specific implementation method of the specific implementation mode is as follows:
the design is mainly developed around a dark channel preoperative algorithm and a CycleGAN-based image defogging algorithm, wherein the dark channel apriori means that in most of the blobs without sky, at least one color channel has some pixel points with very low intensity and close to 0, namely the minimum intensity in the blobs is close to 0. And the CycleGAN is named as: cycle-dependent genetic adaptive Networks, Cycle consensus generates a countermeasure network. The cycle-generated countermeasure network is an unsupervised generation countermeasure network whose main idea is to train two teams of generator-discriminator models to transform images from one domain to another, in the process of which cycle consistency is required.
The method mainly comprises the following steps:
1. construction of a data set and data preprocessing:
the training of the CycleGAN network needs to input pictures of two image domains to be completed into the network, and the quality of data determines the quality of the network training effect, so that foggy image data and clear image data are obtained in the design. Some open source data sets on the network are inquired and processed by combining the data sets established by the user. The open source datasets employed herein are primarily the I-HAZE dataset, the O-HAZE dataset, and the NYU V2 dataset.
2. Architecture of the countermeasure network:
in the complex and diversified structure of the neural network, directional circulation is added in the connection of the recurrent neural network, and the nodes of each layer are not connected when the layers are all connected. The generator G functions to generate a fog-free image from the fog-free image, which may be denoted as G: X → Y, and the generator F functions to generate a fog-free image from the fog-free image, which is generated in the opposite direction to G, which may be denoted as F: Y → X. Generators G and F have the same network structure, but their inputs and outputs are completely opposite. Taking generator G as an example, G is divided into three parts: an Encoding stage (Encoding), a transformer (Transformation) and a Decoding stage (Decoding).
And (3) an encoding stage: the foggy image is taken as input. The main operation in the encoding process is to extract the features of the hazy image through three convolutional layers. Convolution also acts as a downsampling layer in extracting features to continuously reduce the resolution of the feature map. The Convolution layer operations include Convolution (Convolution), Instance Normalization (Instance Normalization) and activation function (Leaky ReLU).
A converter stage: the effect is to combine the features extracted by the encoder stage and use them for conversion from the hazy data domain to the hazless data domain. The transformation stage uses residual blocks to complete the transformation by combining feature maps from the input of the encoding stage with the residual blocks.
And a decoding stage: the decoder is used for recovering low-level features from the feature vectors output in the converter stage to generate fog-free images. The decoder stage restores the original features of the graph using upsampling and finally outputs the final fog-free result by combining the features. And introducing Skip Connection (Skip Connection) between the Encoding module and the Decoding module to transmit data streams so as to provide more information transmission between the Encoding process and the Decoding process, and finally generating a final fog-free image through a convolution and activation function in the Decoding stage.
In cycleGAN there are two discriminators DxAnd Dy. Discriminator DxFor judging whether the input image is a real foggy image or a foggy image generated by the generator F, a discriminator DyFor determining whether the input image is a real fog-free image or a fog-free image generated by the generator G.
3. Data processing and algorithm implementation flow:
the goal of the image defogging training phase is to learn the characteristics of the training data set and save the learned parameters locally. The training phase is mainly divided into the following steps:
3.1, inputting training data and preprocessing the training data;
3.2, defining a generator, a discriminator and a loss function of the defogging network, obtaining globally updatable parameters and initializing global parameters;
3.3 updating Generator G, F and arbiter D in defogging modelx、DyAnd recording the decline of the loss function loss training process.
And 3.4, outputting a fog-free image result, carrying out normalization processing on the fog-free image, and storing the result in the training process.
And 3.5, analyzing test results, and evaluating the quality of the design through qualitative and quantitative evaluation.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (1)
1. An image defogging method based on a generation countermeasure network is characterized in that: the method comprises the following steps:
step one, researching defogging of a dark channel, researching defogging from a background model of fog image imaging, and gradually deriving an estimated value of transmissivity in a derivation model on the basis of an atmospheric scattering model, so that the basic process of fog image imaging and the specific meanings of each parameter in the model are deeply understood, a theoretical cushion is laid for later-stage research, and problems and defects occurring in the algorithm are used as important breakthrough openings of follow-up research;
step two, learning a convolutional neural network to carry out image defogging, estimating atmospheric degradation transmittance by using a DehazeNet model established by a CNN deep structure, and using a bilateral correction linear unit as an activation function to improve the recovery quality of an image;
step three, improving a CycleGAN network, building a neural network and training to complete the conversion from a fog image to a clear image, finally realizing the optimization of the defogging effect, and judging the continuous optimization of various losses through the generation loss, so that the quality of the generated image is continuously improved;
and step four, verifying the defogging effect of the defogged clear image through an experiment, performing qualitative and quantitative evaluation, analyzing a comparison result, and comparing whether the indexes of the improved algorithm based on the generation countermeasure network and the dark channel first-checking algorithm in the aspects of subjective evaluation and objective evaluation are improved or not.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110542673.XA CN113222847A (en) | 2021-05-18 | 2021-05-18 | Image defogging method based on generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110542673.XA CN113222847A (en) | 2021-05-18 | 2021-05-18 | Image defogging method based on generation countermeasure network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113222847A true CN113222847A (en) | 2021-08-06 |
Family
ID=77092747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110542673.XA Pending CN113222847A (en) | 2021-05-18 | 2021-05-18 | Image defogging method based on generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113222847A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115457265A (en) * | 2022-08-25 | 2022-12-09 | 暨南大学 | Image defogging method and system based on generation countermeasure network and multi-scale fusion |
-
2021
- 2021-05-18 CN CN202110542673.XA patent/CN113222847A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115457265A (en) * | 2022-08-25 | 2022-12-09 | 暨南大学 | Image defogging method and system based on generation countermeasure network and multi-scale fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | SFNet-N: An improved SFNet algorithm for semantic segmentation of low-light autonomous driving road scenes | |
| Li et al. | Cross-domain object detection for autonomous driving: A stepwise domain adaptative YOLO approach | |
| CN111539887B (en) | Channel attention mechanism and layered learning neural network image defogging method based on mixed convolution | |
| CN115439857B (en) | Inclined character recognition method based on complex background image | |
| CN109509156B (en) | Image defogging processing method based on generation countermeasure model | |
| Wang et al. | Unsupervised face detection in the dark | |
| CN116311254B (en) | Image target detection method, system and equipment under severe weather condition | |
| CN111680702A (en) | Method for realizing weak supervision image significance detection by using detection frame | |
| Zhang et al. | MSGAN: generative adversarial networks for image seasonal style transfer | |
| Guo et al. | Joint raindrop and haze removal from a single image | |
| CN114627269A (en) | Virtual reality security protection monitoring platform based on degree of depth learning target detection | |
| CN115588237A (en) | Three-dimensional hand posture estimation method based on monocular RGB image | |
| Li et al. | Two‐stage single image dehazing network using swin‐transformer | |
| CN113222847A (en) | Image defogging method based on generation countermeasure network | |
| CN114820733A (en) | Interpretable thermal infrared visible light image registration method and system | |
| Liu et al. | Toward visual quality enhancement of dehazing effect with improved Cycle-GAN | |
| Tian et al. | Semantic segmentation of remote sensing image based on GAN and FCN network model | |
| Yan et al. | Threshold-adaptive unsupervised focal loss for domain adaptation of semantic segmentation | |
| CN115861108A (en) | Image restoration method based on wavelet self-attention generation countermeasure network | |
| Zhang et al. | GAN-based dehazing network with knowledge transferring | |
| Ran et al. | Adaptive fusion and mask refinement instance segmentation network for high resolution remote sensing images | |
| Inoue et al. | Semantic Segmentation of River Video for Efficient River Surveillance System | |
| Zang et al. | Far-infrared object segmentation focus on transmission of overall semantic information | |
| Feng et al. | Coal mine image dust and fog clearing algorithm based on deep learning network | |
| Cheng et al. | Two-stage image dehazing with depth information and cross-scale non-local attention |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |