CN105913392A - Degraded image overall quality improving method in complex environment - Google Patents

Abstract

The invention puts forward a degraded image overall quality improving method in a complex environment and the method is used for solving a problem that poor image quality is caused because degrading factors are not fully overcome in conventional degraded image processing procedures in the complex environment. The degraded image overall quality improving method comprises the following steps: based on a dark channel prior theory, a dark channel of each of a plurality of degraded image frames to be processed is calculated, and therefore a plurality frames of dark channel images are obtained; via use of the plurality frames of obtained dark channel images, each frame of degraded image to be processed is subjected to defogging operation, and a plurality frames of defogged images are obtained; the plurality frames of defogged images are subjected to denoising operation, and therefore a plurality frames of denoised images are obtained; the plurality frames of denoised images are subjected to deblurring operation via a blind restoration method, and therefore a plurality frames of clear images are obtained; the plurality frames of obtained clear images are subjected to super-resolution reconstruction operation based on multiple frame and single frame combination, and finally high-resolution clear images can be obtained. Via use of the degraded image overall quality improving method, obtained images are rich in detailed information, and the degraded image overall quality improving method can be applied to the fields of safety monitoring, traffic detection, safety verification systems and the like.

Description

Degraded image comprehensive quality improving method in complex environment

Technical Field

The invention belongs to the technical field of comprehensive image processing, relates to a quality improvement method for degraded images in a complex environment, and particularly relates to a quality improvement method for comprehensively processing degraded images in a complex environment through defogging, denoising, blind restoration and super-resolution reconstruction, which can be used in the fields of safety monitoring, traffic detection, safety verification systems and the like.

Background

In the current information era, the application of images in various fields is becoming wider and more important is also the requirements for improving the image quality and the resolution. In the process of acquiring images in a complex environment, noise is inevitably introduced due to the influence of objective factors; the image contrast is reduced and the interpretability is deteriorated under the influence of severe weather such as rain, fog and the like; because the shooting environment is severe, the exposure time is required to be increased during shooting, equipment shake is easily caused, image blurring is caused, and the image quality is reduced; the limitation of the performance size of the sensor leads to insufficient resolution of the image, and the requirement for identifying the detail information of the image cannot be met. Therefore, the images shot in the complex environment have the problems of low visibility, high noise, image blurring, low resolution and the like, the comprehensive quality improvement method is used for improving the image quality, and the acquisition of high-resolution and high-quality clear images becomes extremely important.

In the existing image quality improving method, a specific problem is usually solved, but an image shot in a complex environment has multiple degradation factors, if a specific algorithm such as denoising, defogging or deblurring is used for processing the degraded image, only the specific degradation factor aimed by the algorithm can be solved, all degradation problems in the image cannot be solved, and the definition and resolution of a final processing result cannot meet requirements.

Disclosure of Invention

The invention aims to overcome the defects in the prior art, and provides a degraded image comprehensive quality improving method in a complex environment, which is used for solving the technical problem of poor image quality caused by incomplete degradation factor solution in the degraded image processing process in the prior complex environment.

In order to achieve the purpose, the technical scheme adopted by the invention comprises the following steps:

step 1: calculating a dark channel of each frame of degraded image in a plurality of frames of degraded images to be processed according to a dark channel prior theory to obtain a plurality of frames of dark channel images;

step 2: defogging each frame of the multiple frames of degraded images to be processed by utilizing the obtained multiple frames of dark channel images to obtain multiple frames of defogged images;

and step 3: denoising the multiple frames of defogged images to obtain multiple frames of denoised images;

and 4, step 4: deblurring the multiple frames of de-noised images by using a blind restoration method to obtain multiple frames of clear images;

and 5: performing super-resolution reconstruction on the obtained multi-frame clear image, and specifically implementing the following steps:

step 5 a: reconstructing the multi-frame clear image by using a multi-frame super-resolution reconstruction algorithm to obtain a resolution clear image in one frame;

and step 5 b: and performing secondary reconstruction on the resolution clear image in the frame by using a single-frame super-resolution reconstruction algorithm to obtain a final high-resolution clear image.

Compared with the prior art, the invention has the following advantages:

1. the problem of low image visibility is solved through a dark channel prior defogging algorithm, and information submerged in a foggy picture is recovered; denoising by adopting a nonsubsampled contourlet transform NSCT image noise processing method based on the combination of Context model coefficient classification and Bayes adaptive threshold estimation and deblurring by adopting an image restoration method based on sparse constraint, so that image degradation caused by noise pollution and jitter introduced in the shooting process is solved, the image contrast and definition are enhanced, and the image quality of the image is improved; the method of combining the Bayesian-based multi-frame image super-resolution reconstruction algorithm with the dictionary learning-based sparse representation single-frame image super-resolution reconstruction algorithm is adopted to carry out secondary super-resolution reconstruction on the image, so that the image resolution is effectively improved, and finally, a high-resolution clear image is obtained, and more detailed information can be obtained.

2. The invention carries out denoising processing before deblurring processing, effectively solves the problem that a blind restoration algorithm is sensitive to noise, improves the accuracy of image restoration, and increases the definition of the restored image.

3. The blind restoration method of the image adds the operation of automatically selecting the size of the fuzzy kernel, has higher stability compared with the method of manually defining the size of the fuzzy kernel, avoids the phenomena of overlong restoration time and serious ringing effect of the restored image caused by overlarge fuzzy kernel, improves the operation speed, reduces the consumption of the memory of a computer, and does not influence the accurate counting of the fuzzy kernel.

Drawings

FIG. 1 is a block flow diagram of the system of the present invention.

Detailed Description

The technical solution of the present invention is further described below with reference to the accompanying drawings.

With reference to figure 1 of the drawings,

step 1: calculating the dark channel of each frame of degraded image in the multi-frame degraded image to be processed according to the dark channel prior theory,

step 1 a): for a given foggy day shot image j (x), the dark primaries at the pixel points may be expressed as:

$J^{dark}\left(x\right)=\underset{c\∈(R,G,B)}{min}\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{min}\right(J^c\left(y\right)\left)\right)$

omega (x) denotes with point x_iA small square area at the center, c ∈ (R, G, B) represents the color channel of the image, y is a point in Ω (x), J^c(y) represents a pixel value of a y-point,represents a minimum filter;

step 1 b): minimum filtering is performed on the simplified atmosphere model i (x) ═ j (x) t (x) +(1-t (x) a), and the minimum value between three color channels, namely the dark channel of the image, is obtained:

$\underset{c}{min}\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{min}\right(\frac{I^c\left(y\right)}{A^c}\left)\right)=\tilde{t}\left(x\right)\underset{c}{min}\left(\underset{y\∈\mathrm{\Ω}\left(x\right)}{min}\right(\frac{I^c\left(y\right)}{A^c})+(1-\tilde{t}\left(x\right))$

step 2: defogging the degraded image by using the dark channel obtained in the step 1,

step 2 a): rough estimation of transmittance using dark channels already found in step 1

Dark channel J due to image under fog-free condition^darkThe value should approach 0 and be due to the atmospheric light intensity A^cIs usually a relatively large value and is therefore

$\underset{c}{min}\left(\right(\underset{y\∈\mathrm{\Ω}\left(x\right)}{min}\left(\frac{I^c\left(y\right)}{A^c}\right))\→0$

Determining the transmittance of an image

$\tilde{t}\left(x\right)=1-\underset{c}{min}\left(\right(\underset{y\∈\mathrm{\Ω}\left(x\right)}{min}\left(\frac{I^c\left(y\right)}{A^c}\right))$

Step 2 b): and restoring the brightness of the original image by using the optical model, the parameters and the estimated transmissivity of the foggy day image.

Reversely resolving a clear scene image according to an atmospheric scattering model:

$J_{object}=\frac{I^c-A^c}{t\left(x\right)}$

and step 3: aiming at various noises in the image, each frame of the defogged multi-frame image is denoised by adopting a comprehensive noise suppression technology of non-subsampled contourlet transform NSCT,

step 3 a): carrying out multi-scale decomposition on the image through non-subsampled contourlet transformation to obtain sub-band images in all scales and all directions;

step 3 b): using a Context model to count the local correlation of the non-subsampled contourlet transform coefficients and classify the coefficients, using a Bayes self-adaptive method to perform threshold estimation on the classified coefficients in different scale spaces and different directions, and performing noise reduction processing on the image according to the threshold;

step 3 c): and respectively carrying out non-downsampling contourlet inverse transformation on all the processed sub-band images to obtain the noise-suppressed images.

And 4, step 4: and 3, deblurring each frame of the multi-frame de-noised image obtained in the step 3 by utilizing a blind restoration method aiming at the motion blur caused by shaking in the imaging process of a special environment,

step 4 a): determining the layering level according to the blurring degree of the denoised image, and layering the denoised image by utilizing down-sampling to obtain a multi-layer blurred image with the scale from coarse to fine;

step 4 b): because high-frequency information is used during fuzzy kernel estimation, each layer of the multi-layer fuzzy image is preprocessed by utilizing a bilateral filter and an impact filter to obtain a fuzzy image with prominent edges;

step 4 c): blind estimation is carried out on the fuzzy kernel k of the de-noised image by utilizing the obtained strong edge image,

step 4c 1): using discrete filter operatorsFiltering the preprocessed image to obtain a high-frequency part g of the image, and carrying out blind estimation on a fuzzy kernel by using the g, wherein an energy function of a space invariant fuzzy kernel is as follows:

$\underset{x,k}{min}\mathrm{\λ}\left|\right|k*x-g|{|}^2+\frac{\left|\right|x|{|}_1}{\left|\right|x|{|}_2}+\mathrm{\β}|\left|k\right|{|}_1$

and the constraint conditions are met: k is greater than 0, and k is greater than 0,wherein x represents a two-dimensional convolution operation, x is a high-frequency part of an unknown sharp image, k is an unknown blurring kernel, g is a high-frequency part of a preprocessed image, λ is a weight term, and β is a regularization coefficient;

step 4c 2): the multilayer blurred image is changed from a coarse scale to a fine scale, the clear image x and the blurred kernel k are alternately updated in an iterative mode layer by layer, the image estimated in the previous scale and the blurred kernel are sampled and then serve as the input of the next scale until the accurate blurred kernel k is estimated;

target equation when updating sharp image x

$\underset{x}{min}\mathrm{\λ}\left|\right|k*x-g|{|}^2+\frac{\left|\right|x|{|}_1}{\left|\right|x|{|}_2}$

Adopting a fast iterative shrinkage threshold algorithm to solve;

when the blur kernel k is updated, the image x updated last time is taken as a known quantity, so that a reduced energy function can be obtained, and the expression is as follows:

$\underset{x,k}{min}\mathrm{\λ}\left|\right|k*x-g|{|}^2+\mathrm{\β}|\left|k\right|{|}_1$

solving by adopting an unconstrained iteration reweighted least square method;

judging whether the fuzzy core is converged or not by adopting a judgment criterion, wherein the formula is as follows:

$C=abs\left\{\frac{A-B_{dim\left(A\right)\×dim\left(A\right)}}{m}\right\}$

wherein,

the matrices A and B are adjacent scale layers of the fuzzy kernel estimation, B_{dim(A)×dim(A)}The area is a dotted line area in the matrix B, m represents the number of currently decomposed layers, and when C is lower than a set threshold value, the fuzzy kernel is considered to be converged, and the size of the fuzzy kernel is automatically selected;

step 4 d): and deconvoluting the denoised image and the estimated accurate fuzzy kernel k by utilizing a fast deconvolution algorithm of a super-Laplace prior to restore a clear image.

And 5: in order to acquire more image detail information, a mode of combining a multi-frame super-resolution reconstruction technology and a single-frame super-resolution reconstruction technology is adopted to perform super-resolution reconstruction on a multi-frame image obtained after deblurring,

step 5 a): aiming at multi-frame low-resolution images, establishing a mathematical model for the high-resolution images, the image acquisition process, motion parameters and other parameters by using a Bayesian formula, and performing joint estimation on all the parameters through analysis of variational Bayes without parameter adjustment to realize super-resolution image reconstruction of input sequence low-resolution images and obtain the images of resolution in one frame;

step 5 b): the method comprises the steps of continuously carrying out single-frame super-resolution reconstruction on an image obtained by multi-frame image super-resolution reconstruction, carrying out double-dictionary learning by adopting a single-frame image super-resolution reconstruction method based on double sparse dictionaries and utilizing the combination of a K-SVD algorithm and a principal component analysis method dimension reduction method, obtaining a high-resolution dictionary and a low-resolution dictionary at the same time, carrying out sparse coding on the obtained dictionaries through an orthogonal matching tracking algorithm, and obtaining a corresponding high-resolution image block by calculating a sparse coefficient corresponding to the low-resolution dictionary, thereby realizing the super-resolution reconstruction of the image.

The above description and examples are only preferred embodiments of the present invention and should not be construed as limiting the present invention, it will be obvious to those skilled in the art that various modifications and changes in form and detail may be made based on the principle and construction of the present invention after understanding the content and design principle of the present invention, but such modifications and changes based on the inventive concept are still within the scope of the appended claims.

Claims (7)

1. A degraded image comprehensive quality improving method under a complex environment is characterized by comprising the following steps:

1) calculating a dark channel of each frame of degraded image in a plurality of frames of degraded images to be processed according to a dark channel prior theory to obtain a plurality of frames of dark channel images;

2) defogging each frame of the multiple frames of degraded images to be processed by utilizing the obtained multiple frames of dark channel images to obtain multiple frames of defogged images;

3) denoising the multiple frames of defogged images to obtain multiple frames of denoised images;

4) deblurring the multiple frames of de-noised images by using a blind restoration method to obtain multiple frames of clear images;

5) performing super-resolution reconstruction on the obtained multi-frame clear image, and specifically implementing the following steps:

5a) reconstructing the multi-frame clear image by using a multi-frame super-resolution reconstruction algorithm to obtain a resolution clear image in one frame;

5b) and performing secondary reconstruction on the resolution clear image in the frame by using a single-frame super-resolution reconstruction algorithm to obtain a final high-resolution clear image.

2. The method for improving the comprehensive quality of the degraded image in the complex environment according to claim 1, wherein the defogging in the step 2) adopts an image defogging algorithm based on dark channel prior.

3. The method for improving the comprehensive quality of the degraded images in the complex environment according to claim 1, wherein the denoising in the step 3) adopts a non-subsampled contourlet transform NSCT image noise processing method based on the combination of Context model coefficient classification and Bayesian adaptive threshold estimation.

4. The method for improving the comprehensive quality of the degraded images in the complex environment according to claim 1, wherein the deblurring in the step 4) is performed by using an image restoration method based on sparse constraint to deblur each frame of image in the multi-frame de-noised images, and the method is specifically implemented by the following steps:

4a) determining the layering level according to the blurring degree of the denoised image, and layering the denoised image by utilizing down-sampling to obtain a multi-layer blurred image with the scale from coarse to fine;

4b) preprocessing each layer of the multi-layer blurred image by utilizing a bilateral filter and an impact filter to obtain a strong edge image;

4c) blind estimation is carried out on the fuzzy kernel k of the de-noised image by utilizing the obtained strong edge image,

4c1) using discrete filter operatorsFiltering the preprocessed image to obtain a high-frequency part g of the image, and carrying out blind estimation on a fuzzy kernel by using the g, wherein an energy function of a space invariant fuzzy kernel is as follows:

$\underset{x,k}{min}\mathrm{\λ}\left|\right|k*x-g|{|}^2+\frac{\left|\right|x|{|}_1}{\left|\right|x|{|}_2}+\mathrm{\β}|\left|k\right|{|}_1$

and the constraint conditions are met: k is greater than 0, and k is greater than 0,wherein x represents a two-dimensional convolution operation, x is a high-frequency part of an unknown sharp image, k is an unknown blurring kernel, g is a high-frequency part of a preprocessed image, λ is a weight term, and β is a regularization coefficient;

4c2) the multilayer blurred image is changed from a coarse scale to a fine scale, the clear image x and the blurred kernel k are alternately updated in an iterative mode layer by layer, the image estimated in the previous scale and the blurred kernel are sampled and then serve as the input of the next scale until the accurate blurred kernel k is estimated;

4d) and deconvoluting the denoised image and the estimated accurate fuzzy kernel k by utilizing a fast deconvolution algorithm of a super-Laplace prior to restore a clear image.

5. The sparse constraint-based image restoration method according to claim 4, wherein the step 4c2) is implemented by using an objective equation when updating the sharp image x

$\underset{x}{min}\mathrm{\λ}\left|\right|k*x-g|{|}^2+\frac{\left|\right|x|{|}_1}{\left|\right|x|{|}_2}$

And (4) solving by using a fast iterative shrinkage threshold algorithm FISTA.

6. The sparse constraint-based image restoration method according to claim 4, wherein in the step 4c), when performing blind estimation on the blur kernel k of the denoised image, a judgment criterion is used to judge whether the blur kernel converges, and the judgment criterion formula is as follows:

$C=abs\left\{\frac{A-B_{dim\left(A\right)\×dim\left(A\right)}}{m}\right\}$

wherein,

the matrices A and B are adjacent scale layers of the fuzzy kernel estimation, B_{dim(A)×dim(A)}And m represents the number of layers of the current decomposition, and when C is lower than a set threshold value, the fuzzy kernel is considered to be converged, and the size of the fuzzy kernel is automatically selected.

7. The method for improving the comprehensive quality of the degraded images in the complex environment according to claim 1, wherein the multi-frame super-resolution reconstruction in the step 5a) adopts a Bayesian multi-frame image super-resolution reconstruction algorithm, and the single-frame super-resolution reconstruction in the step 5b) adopts a dictionary learning and sparse representation-based single-frame image super-resolution reconstruction algorithm.