CN108564549A - A kind of image defogging method based on multiple dimensioned dense connection network - Google Patents
A kind of image defogging method based on multiple dimensioned dense connection network Download PDFInfo
- Publication number
- CN108564549A CN108564549A CN201810361296.8A CN201810361296A CN108564549A CN 108564549 A CN108564549 A CN 108564549A CN 201810361296 A CN201810361296 A CN 201810361296A CN 108564549 A CN108564549 A CN 108564549A
- Authority
- CN
- China
- Prior art keywords
- image
- defogging
- convolution
- convolutional layer
- multiple dimensioned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 26
- 230000000694 effects Effects 0.000 claims abstract description 24
- 239000003595 mist Substances 0.000 claims abstract description 24
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000004364 calculation method Methods 0.000 claims description 29
- 239000000203 mixture Substances 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 6
- 230000001105 regulatory effect Effects 0.000 claims description 6
- 241000208340 Araliaceae Species 0.000 claims description 5
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims description 5
- 235000003140 Panax quinquefolius Nutrition 0.000 claims description 5
- 235000008434 ginseng Nutrition 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 4
- 238000003475 lamination Methods 0.000 claims description 3
- 238000002156 mixing Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 238000011067 equilibration Methods 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 abstract description 13
- 238000013135 deep learning Methods 0.000 abstract description 9
- 238000012549 training Methods 0.000 abstract description 6
- 230000003044 adaptive effect Effects 0.000 abstract description 4
- 230000000007 visual effect Effects 0.000 abstract description 2
- 230000005540 biological transmission Effects 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 3
- 235000009508 confectionery Nutrition 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012417 linear regression Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 235000019504 cigarettes Nutrition 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000035699 permeability Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration by the use of histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The present invention discloses a kind of image defogging method based on multiple dimensioned dense connection network and greatly improves the quality and visual experience of image by the image reconstruction with different degrees of mist at more clearly image.Be put forward for the first time adaptive histogram equalization mode improve image contrast image preprocessing, significantly improve defog effect；The feature of different scale mist can be described using multiple dimensioned dense connection convolutional neural networks, and is effectively combined its feature, reaches most effective defog effect;Propose the formula based on Retinex defogging problems so that end-to-end deep learning defogging is more succinct effective;The present invention is compared with other defogging algorithms based on deep learning.The present invention not only considerably reduces model parameter quantity, and can also reach ideal defog effect in the case of few training data.
Description
Technical field
The present invention relates to field of image enhancement more particularly to a kind of image defogging sides based on multiple dimensioned dense connection network
Method.
Background technology
Mist is due to a kind of weather phenomenon that small dust and steam particle are assembled and are formed under the dry condition.
Mist, haze, the turbid medias such as cigarette can absorb atmosphere light and cause the scattering of atmosphere light, this leads to the outdoor field acquired under this weather
The image of scape is degenerated.In general, the image degenerated can lose contrast and color fidelity.
By the light of certain scattering medium, the light intensity on former direction can gradually weaken, simultaneously because the conservation of energy is fixed
Rule, the light intensity of decrease can scatter on other directions.The distance of video camera is arrived in addition, scattering lost energy and depending on it.
Based on this physical phenomenon, people commonly use the physical model based on atmospheric scattering to describe the image of mist.There is the image energy of mist
Enough it is expressed as a following linear model：
I (x)=J (x) t (x)+A (1-t (x))
Wherein, I (x) represents the image for having mist, and J (x) is the primary radiation of object, and A is that global atmosphere light is shone, and t (x) is claimed
For medium permeability.However, if only provide the information of single image, while if solving t (x) and A, this is a kind of to owe suitable fixed
The problem of.
In order to solve this problem, most of traditional defogging algorithm is estimated to transmit dependent on hypothesis and priori conditions
Figure, and then other unknown numbers are found out with this.(1) method based on contrast：Compared with the contrast of foggy image, Tan et al.
It was found that the contrast of fog free images is lower, while the variation of its projection ratio is only related with the depth of object, so can using Ma Er
Husband's random field models projection figure[1]；(2) method for the priori that decayed based on color：Zhu et al. uses a kind of based on first
The simple linear regression model (LRM) tested predicts scene depth[2], method is to be utilized to satisfy in the brightness in fogless region and color
Can be closely similar with degree, but have the physical characteristic of very big difference in the brightness of fog bank and color saturation.(3)
Method based on dark：He et al. carries out defogging using the priori of dark.So-called dark is exactly in most non-days
In empty partial zones, the value of light intensity minimum.By dark channel diagram, institute's parameter in need can be obtained from the image for have mist
Value[3].(4) method based on global pixel：Berman et al. is based on a priori：Scheme in one clear clean image
As the quantity of pixel is much larger than the quantity of different colours.Typically for a normal picture, on rgb space, the picture of image
The color of vegetarian refreshments can aggregate into hundreds of small clusters.And these pixels for belonging to the same cluster can be gathered in RGB skies
Between straight line on, these straight lines become mist line.This method exactly estimates transmissivity using mist line, and then passes through atmospherical scattering model
Obtain the image after defogging[4].As can be seen that general defogging algorithm depend heavilys on transmission plot estimation in from the above
Order of accuarcy.And the estimation of transmission plot needs to be based on various priori and hypothesis.Once real image is not met in advance
It is assumed that so the defog effect of image will be excessively poor.
In order to improve the accuracy rate of transmission plot estimation, in recent years, academia starts with deep learning and is gone to solve image
Mist problem.Cai et al. be first proposition with deep learning come learn foggy image to projection figure mapping relations, in this base
Atmospherical scattering model is recycled to reconstruct more clearly image on plinth[5].Ling et al. and Ren et al. improve the estimation of Cai
The method of transmission plot proposes the transmission plot estimation based on depth CNN respectively[6]Estimate with based on multiple dimensioned transmission plot[7].But
It is the problem of this method for separately estimating transmission plot and global air illuminance can lead to suboptimal solution.Because in two parts
The error generated when parameter is estimated respectively will continue to accumulate, and is then amplified when two parameter parallel optimizations.Cause
This, Li et al. people thoroughly converts image defogging problem to end-to-end problem, directly learns foggy image by neural network and arrives
The mapping relations of fog free images[8]。
Although correlative study has been achieved for preferable image defog effect, still there are problems that.Work as use
Traditional mode is come when carrying out image defogging, if the generation of foggy image and the priori conditions or hypothesis of the algorithm are inconsistent
When, its defogging reduced performance can be caused.When using the defogging algorithm based on deep learning, since its robustness is limited to data
Collection, then can be ineffective when handling some images.
Invention content
It is an object of the invention to overcome the deficiencies of the prior art and provide a kind of figures based on multiple dimensioned dense connection network
As defogging method, the image reconstruction of the mist with various concentration can be gone out to more clearly image, significantly improve image
Quality and visual effect.
The technical solution adopted by the present invention is：
A kind of image defogging method based on multiple dimensioned dense connection network comprising following steps：
Step 1, after individual foggy image I (x) being pre-processed, and the image I (x that will be obtained after pretreatment4) conduct
The input data of model；
The step 1 specifically includes following steps：
Step 1.1, image equilibration：Individual foggy image I (x) is passed through into the self-adapting histogram that is limited based on contrast
Equalization method[9]Image preprocessing is carried out, image I (x are obtained1)；
Step 1.2, image normalization：By pretreated foggy image I (x1) each pixel value divided by 255 so that
Each pixel obtains image I (x between [0,1]2)；
Step 1.3, image negative value：By the foggy image I (x after normalized2) each pixel value be multiplied by -1,
So that each pixel between [- 1,0], obtains image I (x3)；
Step 1.4, image positive valueization：By negative valueization treated foggy image I (x3) each pixel add 1, make
Each pixel is obtained between [0,1], obtains image I (x4)；
Step 2, the image I (x that will be obtained after pretreatment4) respectively execute the starting stage, multiple dimensioned convolutional layer calculation stages,
Multiple dimensioned convolution feature mixing calculation stages, variable bring formula calculation stages into, and obtaining a width its corresponding defogging, treated
Image；
The step 2 specifically includes following steps：
Step 2.1, the starting stage：Image I (the x obtained after the pretreatment of input4) carry out convolution algorithm and activation primitive
Operation obtains its corresponding 1st layer of output result F (I (x4)), calculation formula is：
F(I(x4))=max (W*I (x4)+b,0) (1)
Wherein W and b is the convolution weighting parameter and offset parameter of the first layer network of the present invention respectively；
Step 2.2, multiple dimensioned convolutional layer calculation stages：In order to extract different scale mist feature, the present invention will be first
The convolution feature of stage beginning extraction inputs three convolutional layer groups of multiple convolution kernels composition of three kinds of sizes, convolution kernel size point
It Wei not 3x3,5x5 and 7x7.It will be the result F (I (x of starting stage4)) it is separately input to each convolution of multiple dimensioned convolutional layer
Result, is carried out a convolution by layer group again thereafter, and the result that two results in front stack then is input to next convolutional layer.
Its calculation formula is：
WhereinWithIt is the 1st, the 2nd and the 3rd volume of the convolutional layer group of i scales (i=3,5,7) respectively
The convolution weighting parameter of lamination,WithBe respectively the 1st, the 2nd of the convolutional layer group of i scales (i=3,5,7) and
The offset parameter of 3rd convolutional layer.WithIt is the convolution of i scales (i=3,5,7) respectively
The output result of 1st, the 2nd and the 3rd convolutional layer of layer group；
Step 2.3, multiple dimensioned convolution feature mix stages：The present invention feature mix stages by 1 feature stack operation,
2 activation primitive operations and 2 convolution operation groups are at calculation formula is：
F5(I (x))=max (W5*4(I(x))+b5,0) (6)
Wherein W4And W5It is the convolution weights ginseng of the 1st and the 2nd convolutional layer of multiple dimensioned convolution feature mix stages respectively
Number, b4And b5It is the offset parameter of the 1st and the 2nd convolutional layer of multiple dimensioned convolution feature mix stages, F respectively5(I(x))
It is the output of convolutional neural networks of the present invention, i.e., the intermediate variable that convolutional neural networks are learnt；
Step 3, defogging region calculation stages：The intermediate variable that the present invention is learnt by convolutional neural networks, and utilize
This intermediate variable reconstructs more clearly image.
Traditional defogging algorithm is to be based on atmospheric scattering physical model, and formula is as follows：
I (x)=J (x) t (x)+A (1-t (x)) (7)
Wherein, I (x) is the image for having mist, and J (x) is original fogless image, and A is that global atmosphere light is shone, and t (x) is to throw
Penetrate figure.
Since traditional defogging model needs while calculating more parameter, this causes image defogging problem to be owed as one
Well posed problem.Therefore, on the basis of Galdran et al. theories[10], the present invention proposes a kind of from Retinex angle solutions
The certainly formula of defogging problem.Wherein, Galdran et al., which is demonstrated between Retinex theories and image defogging problem, has one kind
Relationship, relationship are as follows：
Dehazing (I (x))=1-Retinex (1-I (x)) (8)
Wherein Dehazing indicates that defogging algorithm, Retinex indicate to carry out the calculation of image defogging based on Retinex theories
Method.The formula identity, the relationship between both methods.
And Retinex theories obey following physical model:
LogR (I (x))=logI (x)-logL (I (x)) (9)
Wherein, L (I (x)) indicates that the reflectogram of foggy image, R (I (x)) indicate to pass through the enhanced figure of Retinex algorithm
Picture.According to above equation, the derivation of equation that defogging problem is solved based on Retinex goes out following calculation formula：
Wherein, ε is regulatory factor, and the present invention chooses ε=0.0001 by experiment, and regulatory factor can be to avoid logarithm
The case where being zero.In conclusion the present invention need to only be optimized by multiple dimensioned dense connection convolutional neural networks in defogging formula
F5(I(x))。
Step 4, by defogging treated image, really clearly image is compared with y, is calculated between two images
Euclidean distance；
Step 5, the Euclidean distance based on calculating, which is constantly updated and optimized, obtains optimal convolution weighting parameter and biasing ginseng
Number；
There is no when the corresponding clearly image of the more clear image of reconstruction is compared preset
Defog effect then continues backpropagation, updates convolution weighting parameter and offset parameter using gradient optimization algorithm, then
Execute step 2-5；
It has been obtained when the corresponding clearly image of the more clear image of recovery is compared preset
When defog effect, then stop backpropagation, and finally acquires convolution weighting parameter and offset parameter that step 2 is obtained.
The present invention uses above technical scheme, in terms of having the advantages that following four：First, multiple dimensioned dense company of the invention
The feature of different scale mist can be described by connecing convolutional neural networks, and be effectively combined its feature, reach most effective defogging
Effect；Second, the present invention proposes the formula based on Retinex defogging problems so that end-to-end deep learning defogging is more
It is succinct effective；Third is compared with other defogging algorithms based on deep learning, and the present invention not only considerably reduces model parameter
Quantity, and can also reach ideal defog effect in the case of few training data；4th, the present invention carries for the first time
Go out adaptive histogram equalization mode improve image contrast image preprocessing, significantly improve defog effect.
Description of the drawings
The present invention is described in further details below in conjunction with the drawings and specific embodiments；
Fig. 1 is that the present invention is based on the whole principle schematics of the image defogging method of multiple dimensioned dense connection network；
Fig. 2 is the principle schematic that the present invention is multiple dimensioned dense connection network；
Fig. 3 is original fog free images；
Fig. 4 is the foggy image formed by fog free images；
Fig. 5 is that AODNet defogging technologies fly treatment effect figure；
Fig. 6 is the treatment effect figure of dark defogging algorithm；
Fig. 7 is using the treatment effect figure of the invention without pre-treatment step；
Fig. 8 is the treatment effect figure using entire protocol of the present invention.
Specific implementation mode
As shown in one of Fig. 1-8, the invention discloses it is a kind of based on it is multiple dimensioned it is dense connection network image defogging method,
It is divided into following steps：
Step 1, the training data preparation stage.
The step 1 specifically includes following steps：
Step 1.1, training dataset is selected.The present invention uses CVPR NTIRE2018 Outdoor Dehaze's
Competition data, it includes foggy images and fog free images pair.Wherein, foggy image is that fog free images are formed by certain algorithm
's.
Step 1.2, image data base is pre-processed, is formed with the pairing of mist subgraph and the fogless subgraph of high definition
Collection.Utilize the adaptive histogram equalization method limited based on contrast[9]Foggy image I (x) is pre-processed, is obtained
Foggy image I (x after setting contrast1).From foggy image I (x1) in by d*d (present invention in d=256) sectional drawing subgraph
Ic, and the interception subgraph J of corresponding size from fog free images J (x) simultaneouslyc, form the pairing set for including N number of subgraph
Step 1.3, image normalization：There is mist subgraph by pretreatedWith fogless subgraphEach pixel value
Divided by 255 so that each pixel is between [0,1]；
Step 1.4, image negative value：There to be mist subgraph after normalizedEach pixel value be multiplied by -1, make
Each pixel is obtained between [- 1,0]；
Step 1.5, image positive valueization：By negative valueization, treated mist subgraphEach pixel add 1 so that
Each pixel is between [0,1]；
Step 2, using depth convolutional neural networks, there is mist subgraph by what is obtained after pretreatmentInput as model
Data execute starting stage, multiple dimensioned convolutional layer calculation stages, multiple dimensioned convolution feature mixing calculation stages, variable band respectively
Enter formula calculation stages, it is final to obtain a width its corresponding defogging treated image；
The step 2 specifically includes following steps：
Step 2.1, the starting stage：Image I (the x obtained after the pretreatment of input4) carry out convolution algorithm and activation primitive
Operation obtains its corresponding 1st layer of output result F (I (x4)), calculation formula is：
F(I(x4))=max (W*I (x4)+b,0) (1)
Wherein W and b is the convolution weighting parameter and offset parameter of the first layer network of the present invention respectively；
Step 2.2, multiple dimensioned convolutional layer calculation stages：In order to extract different scale mist feature, the present invention will be first
The convolution feature of stage beginning extraction inputs three convolutional layer groups of multiple convolution kernels composition of three kinds of sizes, convolution kernel size point
It Wei not 3x3,5x5 and 7x7.It will be the result F (I (x of starting stage4)) it is separately input to each convolution of multiple dimensioned convolutional layer
Result, is carried out a convolution by layer group again thereafter, and the result that two results in front stack then is input to next convolutional layer.
Its calculation formula is：
WhereinWithIt is the 1st, the 2nd and the 3rd volume of the convolutional layer group of i scales (i=3,5,7) respectively
The convolution weighting parameter of lamination,WithBe respectively the 1st, the 2nd of the convolutional layer group of i scales (i=3,5,7) and
The offset parameter of 3rd convolutional layer.WithIt is the convolution of i scales (i=3,5,7) respectively
The output result of 1st, the 2nd and the 3rd convolutional layer of layer group；
Step 2.3, multiple dimensioned convolution feature mix stages：The present invention feature mix stages by 1 feature stack operation,
2 activation primitive operations and 2 convolution operation groups are at calculation formula is：
F5(I (x))=max (W5*4(I(x))+b5,0) (6)
Wherein W4And W5It is the convolution weights ginseng of the 1st and the 2nd convolutional layer of multiple dimensioned convolution feature mix stages respectively
Number, b4And b5It is the offset parameter of the 1st and the 2nd convolutional layer of multiple dimensioned convolution feature mix stages, F respectively5(I(x))
It is the output of convolutional neural networks of the present invention；
Step 3, defogging region calculation stages：The present invention learns intermediate variable by convolutional neural networks, and using in this
Between variable reconstruct more clearly image.
Traditional defogging algorithm is to be based on atmospheric scattering physical model, and formula is as follows：
I (x)=J (x) t (x)+A (1-t (x)) (7)
Wherein, I (x) is the image for having mist, and J (x) is original fogless image, and A is that global atmosphere light is shone, and t (x) is to throw
Penetrate figure.
Since traditional defogging model needs while calculating more parameter, this causes image defogging problem to be owed as one
Well posed problem.Therefore, on the basis of Galdran et al. theories[10], the present invention proposes a kind of from Retinex angle solutions
The certainly formula of defogging problem.Wherein, Galdran et al., which is demonstrated between Retinex theories and image defogging problem, has one kind
Relationship, relationship are as follows：
Dehazing (I (x))=1-Retinex (1-I (x)) (8)
Wherein Dehazing indicates that defogging algorithm, Retinex indicate to carry out the calculation of image defogging based on Retinex theories
Method.The formula identity, the relationship between both methods.
And Retinex theories obey following physical model:
LogR (I (x))=logI (x)-logL (I (x)) (9)
Wherein, L (I (x)) indicates that the reflectogram of foggy image, R (I (x)) indicate to pass through the enhanced figure of Retinex algorithm
Picture.According to above equation, the derivation of equation that defogging problem is solved based on Retinex goes out following calculation formula：
Wherein, ε is regulatory factor, and the present invention chooses ε=0.0001 by experiment, and regulatory factor can be to avoid logarithm
The case where being zero.In conclusion the present invention need to be only optimized in defogging formula by multiple dimensioned dense connection convolutional neural networks
F5(I(x))。
Step 4, by defogging treated image, really clearly image is compared with y, is calculated between two images
Euclidean distance；
Step 5, the Euclidean distance based on calculating, which is constantly updated and optimized, obtains optimal convolution weighting parameter and biasing ginseng
Number；
There is no when the corresponding clearly image of the more clear image of reconstruction is compared preset
Defog effect then continues backpropagation, updates convolution weighting parameter and offset parameter using gradient optimization algorithm, then
Execute step 2-5；
It has been obtained when the corresponding clearly image of the more clear image of recovery is compared preset
When defog effect, then stop backpropagation, and finally acquires convolution weighting parameter and offset parameter that step 2 is obtained.
In order to verify effectiveness of the invention, carried out using the competition data collection of 2018 Outdoor Dehaze of NTIRE
Experiment.The data set includes the image pair of 45 super large resolution ratio.Training dataset is divided into 35 by the present invention, and verification collection is divided into
5, test set is divided into 5.As shown in figures 3-8, the defog effect that the present invention obtains and some existing AODNet[8]With help secretly
Road defogging algorithm[3]It is compared.
The present invention uses Y-PSNR (PSNR:Peak Signal to Noise Ratio) weigh image defogging
Energy.
| Defogging algorithm | AODNet | Dark defogging | (no pretreatment) of the invention | The present invention |
| PSNR | 14.934 | 15.284 | 20.401 | 22.224 |
The PSNR average values of the present invention of table 1 and the prior art in NTIRE2018 Outdoor Dehaze data sets
From table 1 it follows that the PSNR values difference of the present invention classical single image defogging algorithm than in the prior art
Improve 7.29dB, 6.94dB.In addition, from table 1 it can also be seen that using the histogram equalization limited based on contrast
After method is as pretreatment, PSNR average values of the invention rise 1.832dB, to demonstrate this method after pretreatment
Obtain better defog effect.
| Defogging algorithm | C2MSNet | CANDY | The present invention |
| Model parameter number | 3584 | 20990 | 2254 |
2 present invention of table is compared with the model parameter number of the prior art
From Table 2, it can be seen that present invention greatly reduces model parameters.Compared with C2MSNet, model of the invention
Number of parameters reduces 1330, and compared with CANDY, parameter of the invention be only its 1/10th.It can be seen that of the invention
It can also reach ideal defog effect using less parameters.
The novelty of image defogging method proposed by the present invention based on multiple dimensioned dense connection convolutional neural networks is main
It is embodied in four aspects：First, multiple dimensioned dense connection convolutional neural networks of the invention can describe the spy of different scale mist
Sign, and it is effectively combined its feature, reach most effective defog effect；Second, the present invention is proposed based on Retinex defoggings
The formula of problem so that end-to-end deep learning defogging is more succinct effective；Third and other defoggings based on deep learning
Algorithm is compared, and the present invention not only considerably reduces model parameter quantity, and can in the case of few training data,
Also reach ideal defog effect；4th, present invention firstly provides pairs that the mode of adaptive histogram equalization improves image
Than the image preprocessing of degree, defog effect is significantly improved.
Bibliography of the present invention is as follows：
[1]Tan R T.Visibility in bad weather from a single image[C]//Computer
Vision and Pattern Recognition,2008.CVPR 2008.IEEE Conference on.IEEE,2008:1-
8.
[2]Zhu Q,Mai J,Shao L.A Fast Single Image Haze Removal Algorithm
Using Color Attenuation Prior[J].IEEE Trans Image Process,2015,24(11):3522-
3533.
[3]He K,Sun J,Tang X.Single Image Haze Removal Using Dark Channel
Prior.[J]. IEEE Trans Pattern Anal Mach Intell,2011,33(12):2341-2353.
[4]Berman D,Treibitz T,Avidan S.Non-local Image Dehazing[C]//Computer
Vision and Pattern Recognition.IEEE,2016:1674-1682.
[5]Cai B,Xu X,Jia K,et al.DehazeNet:An End-to-End System for Single
Image Haze Removal[J].IEEE Transactions on Image Processing,2016,25(11):5187-
5198.
[6]Ling Z,Fan G,Wang Y,et al.Learning deep transmission network for
single image dehazing[C]//IEEE International Conference on Image
Processing.IEEE, 2016:2296-2300.
[7]Ren W,Liu S,Zhang H,et al.Single Image Dehazing via Multi-scale
Convolutional Neural Networks[M]//Computer Vision–ECCV 2016.Springer
International Publishing,2016:154-169.
[8]Li B,Peng X,Wang Z,et al.AOD-Net:All-in-One Dehazing Network[C]//
IEEE International Conference on Computer Vision.IEEE Computer Society,2017:
4780-4788.
[9]Zuiderveld K.Contrast limited adaptive histogram equalization[M]//
Graphics gems IV.Academic Press Professional,Inc.1994:474-485.
[10]Galdran A,Alvarezgila A,Bria A,et al.On the Duality Between
Retinex and Image Dehazing[J].2017。
Claims (3)
1. a kind of image defogging method based on multiple dimensioned dense connection network, it is characterised in that：It includes the following steps：
Step 1, after individual foggy image I (x) being pre-processed, and the image I (x that will be obtained after pretreatment4) as model
Input data；
The step 1 specifically includes following steps：
Step 1.1, image equilibration：Individual foggy image I (x) is passed through into the self-adapting histogram equilibrium that is limited based on contrast
Change method carries out image preprocessing, obtains image I (x1)；
Step 1.2, image normalization：By pretreated foggy image I (x1) each pixel value divided by 255 so that each
Pixel obtains image I (x between [0,1]2)；
Step 1.3, image negative value：By the foggy image I (x after normalized2) each pixel value be multiplied by -1 so that
Each pixel obtains image I (x between [- 1,0]3)；
Step 1.4, image positive valueization：By negative valueization treated foggy image I (x3) each pixel add 1 so that it is each
Pixel obtains image I (x between [0,1]4)；
Step 2, the image I (x that will be obtained after pretreatment4) starting stage, multiple dimensioned convolutional layer calculation stages, more rulers are executed respectively
Degree convolution feature mixing calculation stages, variable bring formula calculation stages into, obtain a width its corresponding defogging treated image；
The step 2 specifically includes following steps：
Step 2.1, the starting stage：Image I (the x obtained after the pretreatment of input4) carry out convolution algorithm and activation primitive operation and obtain
To its corresponding 1st layer of output result F (I (x4)), calculation formula is：
F(I(x4))=max (W*I (x4)+b,0) (1)
Wherein W and b is the convolution weighting parameter and offset parameter of the first layer network of the present invention respectively；
Step 2.2, multiple dimensioned convolutional layer calculation stages：By the multiple of the convolution feature that the starting stage is extracted three kinds of sizes of input
Three convolutional layer groups of convolution kernel composition；It will be the result F (I (x of starting stage4)) it is separately input to the every of multiple dimensioned convolutional layer
Result is carried out a convolution by a convolutional layer group again thereafter, then the result that two results in front stack is input to next
Convolutional layer.Its calculation formula is：
Wherein WithIt is the convolution power of the 1st, the 2nd and the 3rd convolutional layer of the convolutional layer group of different i scales respectively
Value parameter,WithIt is the biasing ginseng of the 1st, the 2nd and the 3rd convolutional layer of the convolutional layer group of different i scales respectively
Number；WithIt is the 1st, the 2nd and the 3rd volume of the convolutional layer group of different i scales respectively
The output result of lamination；
Step 2.3, multiple dimensioned convolution feature mix stages：Feature mix stages are by 1 feature stack operation, 2 activation primitives
Operation and 2 convolution operation groups are at calculation formula is：
F5(I (x))=max (W5*F4(I(x))+b5,0) (6)
Wherein W4And W5It is the convolution weighting parameter of the 1st and the 2nd convolutional layer of multiple dimensioned convolution feature mix stages respectively,
b4And b5It is the offset parameter of the 1st and the 2nd convolutional layer of multiple dimensioned convolution feature mix stages, F respectively5(I (x)) is volume
The output of product neural network；
Step 3, defogging region calculation stages：It is solved based on Retinex and derives defogging calculation formula：
Wherein, ε is regulatory factor, and I (x) is the image for having mist, and D (X) is clear fogless image；
Step 4, by defogging treated image, really clearly image is compared with y, is calculated European between two images
Distance；
Step 5, the Euclidean distance based on calculating, which is constantly updated and optimized, obtains optimal convolution weighting parameter and offset parameter；
There is no preset defogging when the corresponding clearly image of the more clear image of reconstruction is compared
Effect then continues backpropagation, updates convolution weighting parameter and offset parameter using gradient optimization algorithm, then execute
Step 2-5；
Preset defogging has been obtained when the corresponding clearly image of the more clear image of recovery is compared
When effect, then stop backpropagation, and finally acquires convolution weighting parameter and offset parameter that step 2 is obtained.
2. a kind of image defogging method based on multiple dimensioned dense connection network according to claim 1, it is characterised in that：
The convolution kernel size of three convolutional layer groups is respectively 3x3,5x5 and 7x7, i.e. i=3,5,7 in step 2.2.
3. a kind of image defogging method based on multiple dimensioned dense connection network according to claim 1, it is characterised in that：
Regulatory factor ε values are ε=0.0001 in step 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810361296.8A CN108564549B (en) | 2018-04-20 | 2018-04-20 | Image defogging method based on multi-scale dense connection network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810361296.8A CN108564549B (en) | 2018-04-20 | 2018-04-20 | Image defogging method based on multi-scale dense connection network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108564549A true CN108564549A (en) | 2018-09-21 |
| CN108564549B CN108564549B (en) | 2022-04-05 |
Family
ID=63536010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810361296.8A Active CN108564549B (en) | 2018-04-20 | 2018-04-20 | Image defogging method based on multi-scale dense connection network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108564549B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859120A (en) * | 2019-01-08 | 2019-06-07 | 北京交通大学 | Image defogging method based on multiple dimensioned residual error network |
| CN110097522A (en) * | 2019-05-14 | 2019-08-06 | 燕山大学 | A kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks |
| CN110189262A (en) * | 2019-04-29 | 2019-08-30 | 复旦大学 | Image defogging algorithm based on neural network and Histogram Matching |
| CN110443761A (en) * | 2019-07-16 | 2019-11-12 | 大连理工大学 | A kind of removing rain based on single image method based on multiple dimensioned aggregation features |
| CN110517203A (en) * | 2019-08-30 | 2019-11-29 | 山东工商学院 | A kind of defogging method rebuild based on reference picture |
| CN110544213A (en) * | 2019-08-06 | 2019-12-06 | 天津大学 | Image defogging method based on global and local feature fusion |
| CN110569928A (en) * | 2019-09-23 | 2019-12-13 | 深圳大学 | Micro Doppler radar human body action classification method of convolutional neural network |
| CN110569738A (en) * | 2019-08-15 | 2019-12-13 | 杨春立 | natural scene text detection method, equipment and medium based on dense connection network |
| CN111161360A (en) * | 2019-12-17 | 2020-05-15 | 天津大学 | Retinex theory-based image defogging method for end-to-end network |
| CN112734673A (en) * | 2021-01-13 | 2021-04-30 | 山东大学 | Low-illumination image enhancement method and system based on multi-expression fusion |
| CN113033215A (en) * | 2021-05-18 | 2021-06-25 | 华南师范大学 | Emotion detection method, device, equipment and storage medium |
| CN116681626A (en) * | 2023-08-01 | 2023-09-01 | 山东科技大学 | Target image defogging method based on deep learning in haze weather |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436640A (en) * | 2011-09-21 | 2012-05-02 | 北京航空航天大学 | Foggy-day image sharpening method of multi-scale Retinex model based on HIS space |
| CN105574827A (en) * | 2015-12-17 | 2016-05-11 | 中国科学院深圳先进技术研究院 | Image defogging method and device |
| CN106960425A (en) * | 2017-04-05 | 2017-07-18 | 上海矽奥微电子有限公司 | Single frames defogging method based on multiple dimensioned filtering of deconvoluting |
| US20170213321A1 (en) * | 2016-01-22 | 2017-07-27 | Siemens Healthcare Gmbh | Deep Unfolding Algorithm For Efficient Image Denoising Under Varying Noise Conditions |
-
2018
- 2018-04-20 CN CN201810361296.8A patent/CN108564549B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436640A (en) * | 2011-09-21 | 2012-05-02 | 北京航空航天大学 | Foggy-day image sharpening method of multi-scale Retinex model based on HIS space |
| CN105574827A (en) * | 2015-12-17 | 2016-05-11 | 中国科学院深圳先进技术研究院 | Image defogging method and device |
| US20170213321A1 (en) * | 2016-01-22 | 2017-07-27 | Siemens Healthcare Gmbh | Deep Unfolding Algorithm For Efficient Image Denoising Under Varying Noise Conditions |
| CN106960425A (en) * | 2017-04-05 | 2017-07-18 | 上海矽奥微电子有限公司 | Single frames defogging method based on multiple dimensioned filtering of deconvoluting |
Non-Patent Citations (3)
| Title |
|---|
| DANIEL M PELT等: "A mixed-scale dense convolutional neural network for image analysis", 《PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA》 * |
| 张雷等: "一种新的基于图像增强的融合算法", 《激光与红外》 * |
| 罗会兰等: "一种基于多尺度Retinex算法的图像去雾方法", 《计算机应用与软件》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859120A (en) * | 2019-01-08 | 2019-06-07 | 北京交通大学 | Image defogging method based on multiple dimensioned residual error network |
| CN110189262B (en) * | 2019-04-29 | 2023-03-24 | 复旦大学 | Image defogging method based on neural network and histogram matching |
| CN110189262A (en) * | 2019-04-29 | 2019-08-30 | 复旦大学 | Image defogging algorithm based on neural network and Histogram Matching |
| CN110097522A (en) * | 2019-05-14 | 2019-08-06 | 燕山大学 | A kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks |
| CN110443761A (en) * | 2019-07-16 | 2019-11-12 | 大连理工大学 | A kind of removing rain based on single image method based on multiple dimensioned aggregation features |
| CN110443761B (en) * | 2019-07-16 | 2022-02-15 | 大连理工大学 | Single image rain removing method based on multi-scale aggregation characteristics |
| CN110544213B (en) * | 2019-08-06 | 2023-06-13 | 天津大学 | Image defogging method based on global and local feature fusion |
| CN110544213A (en) * | 2019-08-06 | 2019-12-06 | 天津大学 | Image defogging method based on global and local feature fusion |
| CN110569738A (en) * | 2019-08-15 | 2019-12-13 | 杨春立 | natural scene text detection method, equipment and medium based on dense connection network |
| CN110569738B (en) * | 2019-08-15 | 2023-06-06 | 杨春立 | Natural scene text detection method, equipment and medium based on densely connected network |
| CN110517203B (en) * | 2019-08-30 | 2023-06-23 | 山东工商学院 | Defogging method based on reference image reconstruction |
| CN110517203A (en) * | 2019-08-30 | 2019-11-29 | 山东工商学院 | A kind of defogging method rebuild based on reference picture |
| CN110569928A (en) * | 2019-09-23 | 2019-12-13 | 深圳大学 | Micro Doppler radar human body action classification method of convolutional neural network |
| CN111161360B (en) * | 2019-12-17 | 2023-04-28 | 天津大学 | Image defogging method of end-to-end network based on Retinex theory |
| CN111161360A (en) * | 2019-12-17 | 2020-05-15 | 天津大学 | Retinex theory-based image defogging method for end-to-end network |
| CN112734673B (en) * | 2021-01-13 | 2022-06-21 | 山东大学 | Low-illumination image enhancement method and system based on multi-expression fusion |
| CN112734673A (en) * | 2021-01-13 | 2021-04-30 | 山东大学 | Low-illumination image enhancement method and system based on multi-expression fusion |
| CN113033215B (en) * | 2021-05-18 | 2021-08-13 | 华南师范大学 | Emotion detection method, device, equipment and storage medium |
| CN113033215A (en) * | 2021-05-18 | 2021-06-25 | 华南师范大学 | Emotion detection method, device, equipment and storage medium |
| CN116681626A (en) * | 2023-08-01 | 2023-09-01 | 山东科技大学 | Target image defogging method based on deep learning in haze weather |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108564549B (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564549A (en) | A kind of image defogging method based on multiple dimensioned dense connection network | |
| CN110097519B (en) | Dual-monitoring image defogging method, system, medium and device based on deep learning | |
| CN108230264B (en) | Single image defogging method based on ResNet neural network | |
| CN109410127B (en) | Image denoising method based on deep learning and multi-scale image enhancement | |
| CN106157267B (en) | Image defogging transmissivity optimization method based on dark channel prior | |
| JP6613605B2 (en) | Method and system for restoring depth value of depth image | |
| CN112288658A (en) | Underwater image enhancement method based on multi-residual joint learning | |
| Yin et al. | Highly accurate image reconstruction for multimodal noise suppression using semisupervised learning on big data | |
| CN109377459B (en) | Super-resolution deblurring method of generative confrontation network | |
| CN108416745B (en) | Image self-adaptive defogging enhancement method with color constancy | |
| CN111161360B (en) | Image defogging method of end-to-end network based on Retinex theory | |
| CN109493300B (en) | Aerial image real-time defogging method based on FPGA (field programmable Gate array) convolutional neural network and unmanned aerial vehicle | |
| CN111861906B (en) | Pavement crack image virtual augmentation model establishment and image virtual augmentation method | |
| CN109410144B (en) | End-to-end image defogging processing method based on deep learning | |
| CN110570365A (en) | image defogging method based on prior information | |
| CN110675340A (en) | Single image defogging method and medium based on improved non-local prior | |
| CN110807742B (en) | Low-light-level image enhancement method based on integrated network | |
| CN110097522B (en) | Single outdoor image defogging method based on multi-scale convolution neural network | |
| CN111179196B (en) | Multi-resolution depth network image highlight removing method based on divide-and-conquer | |
| CN105023246B (en) | A kind of image enchancing method based on contrast and structural similarity | |
| CN112419163B (en) | Single image weak supervision defogging method based on priori knowledge and deep learning | |
| CN109712116B (en) | Fault identification method for power transmission line and accessories thereof | |
| CN106023097A (en) | Iterative-method-based flow field image preprocessing algorithm | |
| CN112164010A (en) | Multi-scale fusion convolution neural network image defogging method | |
| Das et al. | Histogram equalization techniques for contrast enhancement: a review |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |