CN114820824A - Real scene vision enhancement method capable of defogging and improving resolution simultaneously - Google Patents

Real scene vision enhancement method capable of defogging and improving resolution simultaneously Download PDF

Info

Publication number
CN114820824A
CN114820824A CN202210507268.9A CN202210507268A CN114820824A CN 114820824 A CN114820824 A CN 114820824A CN 202210507268 A CN202210507268 A CN 202210507268A CN 114820824 A CN114820824 A CN 114820824A
Authority
CN
China
Prior art keywords
image
resolution
prior
defogging
super
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210507268.9A
Other languages
Chinese (zh)
Inventor
张昊
曾铁勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN202210507268.9A priority Critical patent/CN114820824A/en
Publication of CN114820824A publication Critical patent/CN114820824A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/14—Fourier, Walsh or analogous domain transformations, e.g. Laplace, Hilbert, Karhunen-Loeve, transforms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Mathematics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Geometry (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)

Abstract

The visual enhancement method of the real scene capable of removing the haze and improving the resolution simultaneously comprises the steps of firstly obtaining an initial defogging picture and a scattering light picture by using a rank-prior, then combining rank-prior defogging and a super-resolution deep neural network prior for processing various image degradation problems under the framework of a variation model, and simultaneously alternately and iteratively estimating the scattering light picture for removing the haze and seeking a high-quality visual enhancement image through a deep super-resolution network by using an optimization algorithm of semi-quadratic splitting (HQS). The method can also process other complex scenes, and improve the definition and the identifiability of the acquired image.

Description

Real scene vision enhancement method for simultaneously defogging and improving resolution
Technical Field
The invention relates to the technical field of image processing, in particular to a real scene vision enhancement method capable of defogging and improving resolution simultaneously.
Background
When images are captured in severe weather (sand storm or haze) and under different imaging conditions (underwater or low illumination), due to the fact that a large number of particles (such as haze and sand dust) suspended in an environment medium or due to the fact that light rays are subjected to absorption, attenuation and discoloration of water bodies and interference of water impurities in the process of penetrating through the water bodies, the observed images have the problems of low visibility, quality degradation, color distortion and the like, the visibility of the images can be affected by the undesirable degradation, and then subsequent image processing tasks are affected, such as object identification, a road monitoring system, a remote sensing system and the like. Therefore, under complex imaging conditions, such as haze, sand storm, underwater or low light, the visibility enhancement of outdoor images has been an important task for computer vision.
At present, scholars at home and abroad in the field of computer vision propose a plurality of related algorithms for defogging and image super-resolution based on traditional models or neural network training. First, in the field of image defogging, the most notable image defogging method is the image defogging method proposed by hodcamme based on a Dark Channel Prior (DCP) based on the observation that: in most non-sky patches, the intensity of some pixels of at least one color channel is very low, approaching zero in fog-free outdoor images. In addition, many methods for defogging in deep learning networks have been proposed in recent years, among which Cai et al propose a DehazeNet for estimating a medium transmission map, Li et al propose an end-to-end AOD-Net that can directly output a fog-free image, and also some use of a countermeasure generation network (HardGAN) or the like to solve the defogging problem.
Although the above algorithms all obtain good defogging effects, many recovered fog-free images are still damaged by compression artifacts and block artifacts; in addition, noise that is originally present in the original image is amplified along with the defogging process, thereby degrading the clarity and visualization of the final restored image. Considering that the image super-resolution algorithm can utilize image information with higher dimensionality to achieve better image restoration, the image after defogging can be regarded as a low-resolution image, and simultaneously the image resolution is improved by utilizing the image super-resolution method to obtain more details and reduce artifacts and noise.
Single Image Super Resolution (SISR) can be generally classified into the following types: interpolation-based methods, prior-based optimization methods, and neural network-based methods. Among interpolation-based methods, the commonly used ones are bilinear and bicubic interpolators, Lanczos and nearest neighbors, etc., which are convenient but have limited performance in the case of blurring. In the prior-based method, Yang proposes an efficient sparse representation prior for image super-resolution, and the like. Many neural network methods are also used to process low resolution pictures, such as IRCNN, SRMD, SRCNN, VDSR, etc.
Although the above work can achieve good effects in the task of separately processing image defogging or image super-resolution, the obtained restored image either cannot achieve the defogging effect well or is damaged by artifacts and noises, and a clear non-degraded image is difficult to obtain under severe environmental conditions. Therefore, when haze is removed, artifacts and noise in the defogged image are reduced, the image resolution is improved, a clearer visual effect image is obtained, and the technical problem to be solved is solved.
Disclosure of Invention
In order to solve the technical problems, the invention provides a visual enhancement method of a real scene for simultaneously removing haze and improving the image resolution ratio under the framework of a variation model based on rank-prior and a super-resolution depth neural network for processing various image degradation problems as prior, and the method can improve the image resolution ratio while obtaining a defogged image so as to obtain a clearer visual effect.
The invention relates to a real scene vision enhancement method for simultaneously defogging and improving resolution, which comprises the following steps:
s1, inputting a haze image I (x), and obtaining an initial scattered light image based on rank-one prior
Figure BDA0003637874960000023
And (c) aThe initial defogged image j (x);
s2, based on a variational model framework, combining a rank-prior and a super-resolution deep neural network prior to provide an optimization problem;
s3, adding penalty items by introducing auxiliary variables u and v, and obtaining a new unconstrained optimization problem based on a semi-quadratic splitting (HQS) method;
and S4, solving the new unconstrained optimization problem by using an alternative minimization algorithm to obtain the final visual enhancement image.
Further, S1 is, specifically,
s1-1, calculating the unified radiance of each R, G and B channel according to the input haze image I (x)
Figure BDA0003637874960000021
It reflects uniform ambient light:
Figure BDA0003637874960000022
Ω＝m×n,c＝{R,G,B}
wherein, x is the pixel position in the image, c is any one of R, G and B channels, and I c (x) The color channel map of the haze image is shown, m and n are the corresponding length and width of the haze image respectively, and omega is the area size of the haze image.
S2-2 regularization unified radiance
Figure BDA0003637874960000031
Obtaining a unified spectrum characterizing the ambient light source spectrum
Figure BDA0003637874960000032
Figure BDA0003637874960000033
S2-3, obtaining an initial scattered light map according to rank-one prior:
Figure BDA0003637874960000034
get
Figure BDA0003637874960000035
Taking the pixel mean value in I (x) corresponding to the pixel indexes as global atmospheric light A to obtain an initial defogged image J (x) which is as follows:
Figure BDA0003637874960000036
where x is the pixel position in the image, ω ∈ (0, 1)]Is a residual parameter, t 0 Is a small positive number avoiding zero.
Further, in S2, based on the variational model framework, combining rank-one prior and a super-resolution deep neural network prior, the optimization problem proposed is:
Figure BDA0003637874960000037
wherein the content of the first and second substances,
Figure BDA0003637874960000038
represents the convolution operation between the assumed blur kernel k and the high-resolution image J to be restored, ↓ s Is a subsequent down-sampling operation with a scaling factor of s; first term in the formula
Figure BDA0003637874960000039
Is a measure of the scattered light pattern to be updated
Figure BDA00036378749600000310
For the initial scattered light pattern
Figure BDA00036378749600000311
Data fidelity item of (1), second item
Figure BDA00036378749600000312
Is a data fidelity item obtained according to a physical model of defogging, phi (J) is a super-resolution network trained by an SRMD (super-resolution network for multiple definitions), and a Plug-and-Play mode is used as a model prior; alpha and beta are parameters for controlling fidelity terms, mu is a parameter for controlling variable terms, and lambda is a parameter for controlling prior terms of the super-resolution network; here, the first and second liquid crystal display panels are,
Figure BDA00036378749600000313
approximating two discrete differential operators using backward differences with periodic boundary conditions
Figure BDA00036378749600000314
And
Figure BDA00036378749600000315
Figure BDA0003637874960000041
wherein the content of the first and second substances,
Figure BDA0003637874960000042
to represent
Figure BDA0003637874960000043
The pixel value of the ith row and the jth column.
Further, in S3, adding penalty terms by introducing auxiliary variables u and v, and based on an HQS method, obtaining a new unconstrained optimization problem as follows:
Figure BDA0003637874960000044
where γ and σ are penalty parameters, the result of S3 goes to the result of S2 as γ and σ approach infinity.
Further, in S4, the method is advantageousUsing an alternative minimization algorithm to obtain the following sub-problems, and further respectively solving the scattered light images in an iterative manner
Figure BDA0003637874960000045
Auxiliary variables u and v, low resolution image
Figure BDA0003637874960000046
Wherein the content of the first and second substances,
Figure BDA0003637874960000047
is the scatter plot, u, updated at the k +1 iteration k+1 、v k+1 Is an auxiliary variable updated at the (k + 1) th iteration,
Figure BDA0003637874960000048
is the low resolution image updated at iteration k + 1:
Figure BDA0003637874960000049
Figure BDA00036378749600000410
Figure BDA00036378749600000411
s4-1, solving by using a split-Bregman method to obtain
Figure BDA00036378749600000412
And a display solution at the k +1 th iteration based on the x-direction and the y-direction
Figure BDA00036378749600000413
Figure BDA00036378749600000414
Wherein the content of the first and second substances,
Figure BDA00036378749600000415
and
Figure BDA00036378749600000416
respectively a fourier transform and an inverse fourier transform,
Figure BDA00036378749600000417
is to
Figure BDA00036378749600000418
The complex conjugate operator of (a) is,
Figure BDA00036378749600000419
Figure BDA0003637874960000051
here, the
Figure BDA0003637874960000052
And
Figure BDA0003637874960000053
respectively gradient operators based on the x-direction and the y-direction,
Figure BDA0003637874960000054
and
Figure BDA0003637874960000055
respectively, are auxiliary variables that are updated at the k-th iteration based on the x-direction and the y-direction,
Figure BDA0003637874960000056
and
Figure BDA0003637874960000057
parameters updated at the kth iteration based on the x-direction and the y-direction, respectively;
Figure BDA0003637874960000058
wherein the content of the first and second substances,
Figure BDA0003637874960000059
s4-2, directly obtaining a closed-form solution,
Figure BDA00036378749600000510
to obtain a more natural defogging picture, the parameter ω e (0, 1) is introduced, and the above formula can be further written as:
Figure BDA00036378749600000511
s4-3, sub-problem
Figure BDA00036378749600000512
Can be further written as:
Figure BDA00036378749600000513
here, the final visually enhanced image
Figure BDA00036378749600000514
Expressed as a noise level with respect to the low resolution image v, blur kernel k
Figure BDA00036378749600000515
Function of the scaling factor s and the parameter λ:
Figure BDA00036378749600000516
where Θ represents the parameters of MAP inference, the mapping function
Figure BDA00036378749600000517
And learning by a CNN-based super-resolution network SRMD.
The invention has the beneficial effects that: firstly, obtaining an initial defogging picture and a scattering light image by using a rank-one prior, then combining rank-one prior defogging and a super-resolution depth neural network prior for processing various image degradation problems under the framework of a variation model, and simultaneously alternately and iteratively estimating the scattering light image for removing haze and seeking a high-quality vision enhancement image through a depth super-resolution network by using an optimization algorithm of semi-quadratic splitting (HQS); not only can greatly remove the adverse effect of haze, also can reduce the artifact simultaneously and restrain the appearance of noise, finally obtain the detail more clear, visual higher high quality image. In addition, the method can also process other complex scenes, namely image recovery under different weather and imaging conditions, such as sand weather, underwater condition imaging and low-light condition imaging, and has high robustness.
The method can be further applied to other image processing tasks as image processing software, such as a road monitoring system, underwater detection, a monitoring system under the condition of low illumination at night and target identification, and the definition and the identifiability of the acquired image are improved in a complex scene.
Drawings
FIG. 1 is a flow chart of obtaining an initial scattered light image and a defogged image using a rank-one prior;
FIG. 2 is a flow chart of an image visual enhancement algorithm of the present invention combining rank-one prior and a super-resolution depth neural network prior for processing multiple image degradation problems under a variational model;
FIG. 3 shows experimental results of foggy weather, (a) is a foggy weather image, (b) is an image restored by using a dark channel method, (c) is an image restored by directly using rank-one prior, and (d) is a vision enhancement image obtained by the method adopted by the present invention;
FIG. 4 shows the experimental results of sand-dust weather, (a) is the sand-dust weather image, (b) is the image recovered by the famous dark channel method, (c) is the image recovered by directly using rank-one prior, and (d) is the vision enhancement image obtained by the method adopted by the present invention;
fig. 5 shows the results of image restoration experiments under different weather conditions and imaging conditions, wherein (a) is fog weather, (b) is sand weather, (c) is underwater condition imaging, and (d) is low-illumination condition imaging.
Detailed Description
In order that the present invention may be more readily and clearly understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings.
The invention relates to a real scene vision enhancement method for simultaneously defogging and improving resolution, which comprises the following steps:
s1, inputting a haze image I (x), and obtaining an initial scattered light image based on rank-one prior
Figure BDA0003637874960000061
And an initial defogged image j (x);
mathematical models describing the degradation of light due to absorption and scattering effects caused by particles in the atmosphere are generally:
Figure BDA0003637874960000071
where x is the pixel position in the image, o denotes dot product, i (x) and j (x) are the image to be defogged and the image to be restored, respectively, a is the global atmospheric light, and t (x) is the air refractive index.
The invention does not consider solving for the refractive index t (x) of air, but, instead, solves for a scattered light map describing the portion of ambient scattered light that affects imaging
Figure BDA0003637874960000072
Further, can be made of
Figure BDA0003637874960000073
The rewriting model is solved to obtain a fog-free image to be restored J (x):
Figure BDA0003637874960000074
wherein ω ∈ (0, 1)]Is a residual parameter that in practice maintains a small amount of haze, t, for distant objects 0 Is a small positive number avoiding zero, t 0 0.001; based on the above model, in order to obtain the recovered defogged image, the scattered light image needs to be solved
Figure BDA0003637874960000075
And global atmospheric light a.
As shown in fig. 1, the specific steps of S1 are:
s1-1, calculating the unified radiance of each R, G and B channel according to the input haze image I (x)
Figure BDA0003637874960000076
It reflects uniform ambient light:
Figure BDA0003637874960000077
Ω＝m×n,c＝{R,G,B}
wherein m and n are the size of the image;
s2-2 regularization unified radiance
Figure BDA0003637874960000078
Obtaining a unified spectrum characterizing the ambient light source spectrum
Figure BDA0003637874960000079
Figure BDA00036378749600000710
S2-3, obtaining an initial scattered light map according to rank-one prior:
Figure BDA00036378749600000711
get
Figure BDA00036378749600000712
Taking the pixel mean value in I (x) corresponding to the pixel indexes as global atmospheric light A to obtain an initial defogged image J (x) which is as follows:
Figure BDA0003637874960000081
where x is the pixel position in the image, ω ∈ (0, 1)]Is a residual parameter, t 0 Is a small positive number avoiding zero.
S2, based on a variational model framework, combining a rank-prior and a super-resolution deep neural network prior to provide an optimization problem;
after obtaining the initial scattered light pattern
Figure BDA0003637874960000082
After the initial defogged image J (x), the invention provides the following optimization problems through a variational model framework and combining rank-prior defogging and a super-resolution deep neural network SRMD prior for processing various image degradation problems:
Figure BDA0003637874960000083
wherein the content of the first and second substances,
Figure BDA0003637874960000084
represents the convolution operation between the assumed blur kernel k and the high-resolution image J to be restored, ↓ s Is a subsequent down-sampling operation with a scaling factor of s; the first term in the formula is a measure
Figure BDA00036378749600000811
For the
Figure BDA00036378749600000812
The second item is a data fidelity item obtained according to a defogging physical model, phi (J) is a super-resolution network trained by an SRMD (super-resolution network for multiple resolutions), and a Plug-and-Play mode is used as a model prior; alpha and beta are parameters for controlling the fidelity term, mu is a parameter for controlling the variation term, and lambda is a parameter for controlling the prior term of the super-resolution network. Here, the first and second liquid crystal display panels are,
Figure BDA0003637874960000085
approximating two discrete differential operators using backward differences with periodic boundary conditions
Figure BDA0003637874960000086
And
Figure BDA0003637874960000087
Figure BDA0003637874960000088
wherein the content of the first and second substances,
Figure BDA0003637874960000089
to represent
Figure BDA00036378749600000810
The pixel value of the ith row and the jth column.
S3, adding penalty items by introducing auxiliary variables u and v, and obtaining a new unconstrained optimization problem based on a semi-quadratic splitting (HQS) method;
Figure BDA0003637874960000091
where γ and σ are penalty parameters, the result of equation (4) tends towards equation (3) as γ and σ approach infinity.
S4, solving a new unconstrained optimization problem by using an alternative minimization algorithm to obtain a final visual enhancement image; the method specifically comprises the following steps:
by using the alternative minimization algorithm, the following sub-problems can be obtained, and then the scattering light images are respectively solved in an iterative manner
Figure BDA0003637874960000092
Auxiliary variables u and v, low resolution image
Figure BDA0003637874960000093
Figure BDA0003637874960000094
Figure BDA0003637874960000095
Figure BDA0003637874960000096
Solving (5) can be realized by utilizing a split-Bregman method,
Figure BDA0003637874960000097
and
Figure BDA0003637874960000098
alternative solution
Figure BDA0003637874960000099
And (u) x ，u y ):
Figure BDA0003637874960000101
Figure BDA0003637874960000102
Since the right side of equation (9) is differentiable, using a Fast Fourier Transform (FFT), one can obtain
Figure BDA0003637874960000103
As a result of (1):
Figure BDA0003637874960000104
wherein the content of the first and second substances,
Figure BDA0003637874960000105
Figure BDA0003637874960000106
solving the formula (10) by using a generalized contraction formula to obtain
Figure BDA0003637874960000107
The display solution of (1):
Figure BDA0003637874960000108
wherein the content of the first and second substances,
Figure BDA0003637874960000109
and (6) directly solving a closed-form solution,
Figure BDA00036378749600001010
to get a more natural defogged image, similar to model (2), the parameter ω ∈ (0, 1) is introduced, and equation (13) can be further written as:
Figure BDA00036378749600001011
solving (7) equation (7) can be further written first
Figure BDA00036378749600001012
Here, the final visually enhanced image
Figure BDA00036378749600001013
Expressed as a noise level with respect to the low resolution image v, blur kernel k
Figure BDA0003637874960000111
Function of the scaling factor s and the parameter λ:
Figure BDA0003637874960000112
where Θ represents the parameters of MAP inference, the mapping function
Figure BDA0003637874960000113
And learning by a CNN-based super-resolution network SRMD.
So far, according to the iterative optimization solving process, the optimized scattered light image can be obtained
Figure BDA0003637874960000114
And final defogged and resolution enhanced visual enhancement image
Figure BDA0003637874960000115
As shown in fig. 3, in haze weather, compared with the famous dark channel prior method, wherein the method of the present invention is shown in fig. 3(d), when processing sky and non-sky regions, block artifacts and fog do not occur, and the details of the recovered image are clearer, while in fig. 3(b), the dark channel prior generates undesirable block artifacts and noise during processing, and the overall effect of the image is blurred. In addition, compared with an image directly restored by only using rank-one prior, the method greatly removes haze, reduces artifacts and inhibits the occurrence of noise.
Fig. 4 shows that in a sand-dust weather, the dark channel prior method is as shown in fig. 4(b), sand and dust cannot be removed well, and an image restored directly by using rank-one prior is as shown in fig. 4(c), although sand and dust can be removed, noise existing in an original image is amplified at the same time, whereas the method of the present invention is as shown in fig. 4(d), and the occurrence of noise is suppressed while sand and dust are removed, so that an obtained restored image is clearer.
Fig. 5 shows the robustness of the present invention, and the method of the present invention can still work well and recover more details when dealing with degraded images under different weather and conditions.
The variation model not only adds a total variation term, but also adds a rank-one prior and a super-resolution depth neural network prior into the same optimization problem, wherein the total variation term can effectively remove undesired noise in the image, and the image is smoothed by gradient descent, so that the image is effectively smoothed in the image, and a relatively sharp edge is obtained at the edge of the image as much as possible; the rank-one prior can effectively remove the adverse effects of haze, dust, sand and the like in the image; the super-resolution deep neural network can effectively remove adverse effects of compression artifacts, block artifacts, noise and the like, and enhances the visibility of pictures. Based on the above, the visual enhancement method for the real scene, which is provided under the framework of the variational model and can remove haze and improve the resolution of the image, can improve the resolution of the image while obtaining the defogged image, thereby obtaining a clearer visual effect.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and all equivalent variations made by using the contents of the present specification and the drawings are within the protection scope of the present invention.

Claims (5)

1. A visual enhancement method for a real scene with defogging and resolution enhancement simultaneously is characterized by comprising the following steps:
s1, inputting a haze image I (x), and obtaining an initial scattered light image based on rank-one prior
Figure FDA0003637874950000011
And an initial defogged image j (x);
s2, based on a variational model framework, combining a rank-prior and a super-resolution deep neural network prior to provide an optimization problem;
s3, adding penalty items by introducing auxiliary variables u and v, and obtaining a new unconstrained optimization problem based on a semi-quadratic splitting method;
and S4, solving the new unconstrained optimization problem by using an alternative minimization algorithm to obtain the final visual enhancement image.
2. The visual enhancement method for a real scene with defogging and resolution enhancement functions according to claim 1, wherein the step S1 is specifically as follows:
s1-1, calculating the unified radiance of each R, G and B channel according to the input haze image I (x)
Figure FDA0003637874950000012
Figure FDA0003637874950000013
Ω＝m×n，c＝{R，G，B}
Wherein, x is the pixel position in the image, c is any one of R, G and B channels, and I c (x) The image is a color channel image of the haze image, m and n are the length and width corresponding to the haze image respectively, and omega is the area size of the haze image;
s2-2 regularization unified radiance
Figure FDA0003637874950000014
Obtaining a unified spectrum characterizing the ambient light source spectrum
Figure FDA0003637874950000015
Figure FDA0003637874950000016
S2-3, obtaining an initial scattered light map according to rank-one prior:
Figure FDA0003637874950000017
get
Figure FDA0003637874950000018
Taking the pixel mean value in I (x) corresponding to the pixel indexes as global atmospheric light A to obtain an initial defogged image J (x) which is as follows:
Figure FDA0003637874950000019
where x is the pixel position in the image, ω ∈ (0, 1)]Is a residual parameter, t 0 Is a small positive number avoiding zero.
3. The method of claim 1, wherein in S2, based on the variational model framework, in combination with the rank-one prior and a super-resolution deep neural network prior, the optimization problem is:
Figure FDA0003637874950000021
wherein the content of the first and second substances,
Figure FDA0003637874950000022
represents the convolution operation between the assumed blur kernel k and the high-resolution image J to be restored, ↓ s Is a subsequent down-sampling operation with a scaling factor of s; first term in the formula
Figure FDA0003637874950000023
Is a measure of the scattered light pattern to be updated
Figure FDA0003637874950000024
For the initial scattered light pattern
Figure FDA0003637874950000025
Data fidelity item of (1), second item
Figure FDA0003637874950000026
Is a data fidelity item obtained according to a defogging physical model, phi (J) is a super-resolution network trained by SRMD, and a Plug-and-Play mode is used as model prior; alpha and beta are parameters for controlling fidelity terms, mu is a parameter for controlling variable terms, and lambda is a parameter for controlling prior terms of the super-resolution network; here, the first and second liquid crystal display panels are,
Figure FDA0003637874950000027
approximating two discrete differential operators using backward differences with periodic boundary conditions
Figure FDA0003637874950000028
And
Figure FDA0003637874950000029
Figure FDA00036378749500000210
wherein the content of the first and second substances,
Figure FDA00036378749500000211
to represent
Figure FDA00036378749500000212
The pixel value of the ith row and the jth column.
4. The visual enhancement method for real scenes with defogging and resolution enhancement simultaneously as claimed in claim 1, wherein in S3, penalty term is added by introducing auxiliary variables u and v, and based on HQS method, new unconstrained optimization problem is obtained as follows:
Figure FDA00036378749500000213
where γ and σ are penalty parameters, the result of S3 goes to the result of S2 as γ and σ approach infinity.
5. The method of claim 1, wherein in step S4, the following sub-problems are obtained by using the alternative minimization algorithm, and the scattered light patterns are solved iteratively and respectively
Figure FDA0003637874950000031
Auxiliary variables u and v, low resolution image
Figure FDA0003637874950000032
Wherein the content of the first and second substances,
Figure FDA0003637874950000033
is the scatter plot, u, updated at the k +1 iteration k+1 、v k+1 Is an auxiliary variable updated at the (k + 1) th iteration,
Figure FDA0003637874950000034
is the (k + 1) th iterationUpdated low resolution image:
Figure FDA0003637874950000035
Figure FDA0003637874950000036
Figure FDA0003637874950000037
s4-1, solving by using a split-Bregman method to obtain
Figure FDA0003637874950000038
And a display solution at the k +1 th iteration based on the x-direction and the y-direction
Figure FDA0003637874950000039
Figure FDA00036378749500000310
Wherein the content of the first and second substances,
Figure FDA00036378749500000311
and
Figure FDA00036378749500000312
respectively a fourier transform and an inverse fourier transform,
Figure FDA00036378749500000313
is to
Figure FDA00036378749500000314
The complex conjugate operator of (a) is,
Figure FDA00036378749500000315
Figure FDA00036378749500000316
here, the
Figure FDA00036378749500000317
And
Figure FDA00036378749500000318
respectively based on gradient operators in the x-direction and in the y-direction,
Figure FDA00036378749500000319
and
Figure FDA00036378749500000320
respectively, are auxiliary variables that are updated at the k-th iteration based on the x-direction and the y-direction,
Figure FDA00036378749500000321
and
Figure FDA00036378749500000322
parameters updated at the kth iteration based on the x-direction and the y-direction, respectively;
Figure FDA00036378749500000323
wherein the content of the first and second substances,
Figure FDA00036378749500000324
s4-2, directly obtaining a closed-form solution,
Figure FDA0003637874950000041
to obtain a more natural defogging picture, the parameter ω e (0, 1) is introduced, and the above formula can be further written as:
Figure FDA0003637874950000042
s4-3, sub-problem
Figure FDA0003637874950000043
Can be further written as:
Figure FDA0003637874950000044
here, the final visually enhanced image
Figure FDA0003637874950000045
Expressed as a function of the low resolution image v, the blur kernel k, the noise level
Figure FDA0003637874950000046
Function of the scaling factor s and the parameter λ:
Figure FDA0003637874950000047
where Θ represents the parameters of MAP inference, the mapping function
Figure FDA0003637874950000048
And learning by a CNN-based super-resolution network SRMD.
CN202210507268.9A 2022-05-11 2022-05-11 Real scene vision enhancement method capable of defogging and improving resolution simultaneously Pending CN114820824A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210507268.9A CN114820824A (en) 2022-05-11 2022-05-11 Real scene vision enhancement method capable of defogging and improving resolution simultaneously

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210507268.9A CN114820824A (en) 2022-05-11 2022-05-11 Real scene vision enhancement method capable of defogging and improving resolution simultaneously

Publications (1)

Publication Number Publication Date
CN114820824A true CN114820824A (en) 2022-07-29

Family

ID=82514361

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210507268.9A Pending CN114820824A (en) 2022-05-11 2022-05-11 Real scene vision enhancement method capable of defogging and improving resolution simultaneously

Country Status (1)

Country Link
CN (1) CN114820824A (en)

Similar Documents

Publication Publication Date Title
Wang et al. Dehazing for images with large sky region
CN102326379B (en) Method for removing blur from image
CN109118446B (en) Underwater image restoration and denoising method
CN116579945B (en) Night image restoration method based on diffusion model
CN115222614A (en) Priori-guided multi-degradation-characteristic night light remote sensing image quality improving method
Honnutagi et al. Fusion-based underwater image enhancement by weight map techniques
CN112907461B (en) Defogging enhancement method for infrared foggy-day degraded image
Zhu et al. Low-light image enhancement network with decomposition and adaptive information fusion
Van Nguyen et al. Real-time image and video dehazing based on multiscale guided filtering
Cai et al. Underwater image processing system for image enhancement and restoration
Wang et al. A convex single image dehazing model via sparse dark channel prior
Li et al. DLT-Net: deep learning transmittance network for single image haze removal
Du et al. A new image decomposition approach using pixel-wise analysis sparsity model
Wang et al. Rapid nighttime haze removal with color-gray layer decomposition
Liu et al. Joint dehazing and denoising for single nighttime image via multi-scale decomposition
CN114820824A (en) Real scene vision enhancement method capable of defogging and improving resolution simultaneously
Wu et al. Computational ghost imaging with 4-step iterative rank minimization
KR102342940B1 (en) Method for One-Step L0 Smoothing via Deep Gradient Prior
Gautam et al. A model-based dehazing scheme for unmanned aerial vehicle system using radiance boundary constraint and graph model
Unnikrishnan et al. Non-Local Retinex Based Dehazing and Low Light Enhancement of Images.
Bull Super-resolution of satellite imagery
Pal et al. Quality enhancement of foggy images comprising of large sky region on SAMEER TU dataset
Wu et al. MSDB-based CNN architecture for image dehazing in driverless cars
Roy Single Image DnCNN Visibility Improvement (SImDnCNNVI)
Zhang et al. Image repair and recovery

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination